Python - OBAFGKM https://www.obafgkm.ca/category/python/ Astro/Geo/Data Tue, 04 Feb 2020 16:56:53 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.4 https://i0.wp.com/www.obafgkm.ca/wp-content/uploads/2022/04/cropped-RMS-headbadge-colour-notext.png?fit=32%2C32&ssl=1 Python - OBAFGKM https://www.obafgkm.ca/category/python/ 32 32 204991942 Building an inexpensive lux meter for the observatory… https://www.obafgkm.ca/2020/02/04/cheap-light-meters-for-the-observatory/?utm_source=rss&utm_medium=rss&utm_campaign=cheap-light-meters-for-the-observatory https://www.obafgkm.ca/2020/02/04/cheap-light-meters-for-the-observatory/#respond Tue, 04 Feb 2020 16:56:53 +0000 http://orebits.ca/?p=463 As part of an environmental monitoring system for an observatory, I wanted to include a measure of ambient light. Ostensibly, this is for safety reasons. However, I must admit that I really just want to see if I can (coarsely) measure sky brightness. There are a number of inexpensive light sensors available from AliExpress/Amazon. The two that I show here are based on the TEMT6000 sensor and the LM393 voltage comparator. Both cost less than $1.50. Hooking up the sensors…

The post Building an inexpensive lux meter for the observatory… first appeared on OBAFGKM.

The post Building an inexpensive lux meter for the observatory… appeared first on OBAFGKM.

]]>
As part of an environmental monitoring system for an observatory, I wanted to include a measure of ambient light. Ostensibly, this is for safety reasons. However, I must admit that I really just want to see if I can (coarsely) measure sky brightness.

There are a number of inexpensive light sensors available from AliExpress/Amazon. The two that I show here are based on the TEMT6000 sensor and the LM393 voltage comparator. Both cost less than $1.50.

Hooking up the sensors is really quite straightforward. Connect the GND pin to GND, the VCC pin to 3.3V, and the A0/S pin to analog pin 0. You’re done.

A simple Arduino test script would look like…

const int numReadings = 10;
int reading;
int total = 0;
int average = 0;

int lightPin = 0;
float lux;

void setup() {
  Serial.begin(9600);
}

void loop() {
  for (int i = 0; i < numReadings; i++){
    reading = analogRead(lightPin);
    total = total + reading;
    Serial.println(total);
    delay(100);
  }
  average = total / numReadings;
  total = 0;
  lux = pow(10,6.6162) * pow(average,-1.9191);
  Serial.print("Reading: "); Serial.print(average);
  Serial.print(" ==> "); Serial.print(lux); Serial.println(" lux");
  delay(1000);        // delay in between reads for stability
}

 

Upload the sketch to your Arduino and open up a serial monitor. You should see values from 0 to 1023. If you’ve got a TEMT6000 hooked up the numbers will increase with light levels while an LM393-based device will show readings that decrease with light levels.

At first glance, it looks like the TEMT6000 does not have sufficient sensitivity/resolution at low light levels, so it looks like the LM393 will be used for observatory monitoring.

The values coming out of the sensor give a relative measure of ambient light. But, what we’d really like is an absolute measure of light levels measured in lux. The best way to do this is to measure sensor readings at a number of different known light levels. To get the ‘known’ light levels, you can use a light meter that measures in lux. These are available cheaply (although you get what you pay for) from AliExpress and are likely to be much better than just ‘winging’ it. To calibrate…

  1. Setup in a room that you can adjust the lighting from low/dark to bright.
  2. Take measurements measurements from both the lux meter and your Arduino light meter at, say, 10 intervals from dark to full brightness. You could tape the LM393 to the side of the light meter so that they’re both ‘looking’ in roughly the same direction.
  3. Move outside and take more measurements. Sample in shady spots, bright spots, and full-on sun if you can.
  4. Plot your measurements as known (lux meter) vs measured (Arduino) as a log-log plot.
  5. Find a best-fit line to your log-log plot in the form, y = m * x+ b, where y is the known level, x is the measured level, and m & b are the slope & intercept of the best-fit line
  6. Your transformation from measured to known is, lux = xm*10b

For my sensor, the transformation equation works out to be lux = reading-1.9191x106.6162

What’s next? Well, now that we have the basics of the lux meter, it can be incorporated into other projects.

The post Building an inexpensive lux meter for the observatory… first appeared on OBAFGKM.

The post Building an inexpensive lux meter for the observatory… appeared first on OBAFGKM.

]]>
https://www.obafgkm.ca/2020/02/04/cheap-light-meters-for-the-observatory/feed/ 0 463
How to find stellar magnitude limit – A bit of a hack https://www.obafgkm.ca/2019/12/05/how-to-find-stellar-magnitude-limit-a-bit-of-a-hack/?utm_source=rss&utm_medium=rss&utm_campaign=how-to-find-stellar-magnitude-limit-a-bit-of-a-hack https://www.obafgkm.ca/2019/12/05/how-to-find-stellar-magnitude-limit-a-bit-of-a-hack/#respond Thu, 05 Dec 2019 00:41:21 +0000 http://orebits.ca/?p=143 Finding the stellar magnitude limit in an image is not necessarily as straightforward as it, at first, appears. I needed to come up with some non-hand-wavy numbers and thought, ‘No problem, this is easy.’ It turns out that I was wrong. At least for my imagery and what I wanted to say about it. The ‘proper’ way to calculate the limiting magnitude, it turns out, is to inject synthetic stars of different magnitudes into your imagery and then try to…

The post How to find stellar magnitude limit – A bit of a hack first appeared on OBAFGKM.

The post How to find stellar magnitude limit – A bit of a hack appeared first on OBAFGKM.

]]>
Finding the stellar magnitude limit in an image is not necessarily as straightforward as it, at first, appears. I needed to come up with some non-hand-wavy numbers and thought, ‘No problem, this is easy.’ It turns out that I was wrong. At least for my imagery and what I wanted to say about it.

The ‘proper’ way to calculate the limiting magnitude, it turns out, is to inject synthetic stars of different magnitudes into your imagery and then try to extract them. The magnitude cutoff at which you’re only able to detect 1/2 of the stars at that magnitude is your limiting magnitude. Easy with IRAF (he says with a grin). After struggling mentally with it for a bit, I realized that it wasn’t going to be easy (nor peasy, nor lemon squeezy) at all, so I’d look for another way.

The images that I wanted to work with were from the EMCCD cameras that are collecting the data for my PhD project. They’re of a moderate field-of-view (ca. 15°x15°) and tend to have lots of stars. In a quick visual scan, one can pick out stars (sometimes) down to 12th magnitude. Not bad for a 50 mm photographic lens running at 32 fps. But is that the limiting magnitude? As it turns out, nope.

The wavelength response of the unfiltered EMCCD system is fairly wide. It goes from about 375 nm out to around a μm (the lens is not responsive beyond about 950 nm), so tying it to a particular filter is challenging. Which means that we can’t really compare our measured magnitudes directly to, say, the V magnitudes that might be given in our comparison catalog. Ideally, we’d look for a comparison catalog that gives magnitudes in a similar passband as our system. It turns out that the Gaia G passband is a reasonably close match.

The procedure

  1. Correct your image for bias, dark, and flat
  2. Find the plate solution for your image (try Astrometry.net or SCAMP)
  3. Run SExtractor on the image to create a catalog of sources
  4. Run some python code to cross-check the detected objects against Gaia DR2
  5. Look at the plot of instrumental vs Gaia magnitudes and decide where real object detections falls off and call that your ‘limit’

The code will provide a handy plot showing instrumental magnitude vs Gaia G-band magnitude. What you’ll see is a trend that drops off at some magnitude. The point at which it drops off is the ultimate limiting magnitude.

Setting up SCAMP and SExtractor

More to come…

Downloading Gaia

When I first did this exercise I queried (using astroquery) the Gaia DR2 online database for each detected source separately. With a couple of thousand sources in each image, this can take a lot of time. A better solution is to download a larger area of the catalog and then query that directly. There are a number of different places to do this, but the one that I used can be found here…

http://jvo.nao.ac.jp/portal/gaia/dr2.do

As the first step of the process you select the fields that you want to retrieve. I chose: source_id, ra, dec, parallax (no sure why I bothered), phot_g_mean_mag, phot_bp_mean_mag, and phot_rp_mean_mag. Next, I chose a field radius of 45 degrees centered on an RA/Dec of 0/90 and proceeded with the download.

import matplotlib.pyplot as plt
import sys
import pandas as pd

filename = 'test.cat'
# filename = sys.argv[1]
gfilename = "./GaiaDR2_45to90-M13.psv"

SR = 0.03 # Search distance in degrees

data = pd.read_csv(filename, delim_whitespace=True, skiprows=14)
data.columns = ['Number','XWIN','YWIN','XWORLD','YWORLD','MAG','MAGERROR','FLUX','FLUXERROR','FLUXRADIUS','FWHM','BACKGROUND','RA','DEC']

gdata = pd.read_csv(gfilename, delimiter="|", skiprows=4, low_memory=False)
gdata.columns = ["ID","gRA","gDEC","PAR","gMAG","bMAG","rMAG"] # Make sure that these columns are the ones downloaded from the data server

gdata = gdata.drop('ID', axis=1) # May as well get rid of the ID column

data['BMAG'] = 99
data['GMAG'] = 99
data['RMAG'] = 99

for i in range(len(data.index)):
    RA = data.loc[i,'RA']
    DEC = data.loc[i,'DEC']

    df = gdata[(gdata.gRA < (RA+SR))]
    df = df[(df.gRA > (RA-SR))]
    df = df[(df.gDEC < (DEC+SR))]
    df = df[(df.gDEC > (DEC-SR))]
    df.sort_values(by=["gMAG"], ascending=True, inplace=True)
    df.reset_index(drop=True, inplace=True)

    if len(df.index)>1:
        data.loc[i,'BMAG'] = df.loc[0]['bMAG']
        data.loc[i,'GMAG'] = df.loc[0]['gMAG']
        data.loc[i,'RMAG'] = df.loc[0]['rMAG']

print('Pickling...')
data.to_pickle('out.pkl')
print('Pickled...')

print('Plotting figure...')

plt.figure(figsize=(14,10))

plt.scatter(data['GMAG'],data['MAG'], color='green')

plt.xlabel('Gaia DR2 G-band Magnitude', size=15)
plt.ylabel('Instrumental Magnitude', size=15)
plt.xlim(4,13)
plt.ylim(-15,-2)
plt.tight_layout()
plt.savefig('Imag-vs-Gmag.png', dpi=300)

plt.show()

The above code will save the results in a .pkl file and produce a rudimentary plot showing the SExtractor instrumental magnitude plotted against the Gaia g-band magnitude. Since you’ve got the pickle file, you can always play with it a bit and make a more informative plot like the one below. The example data given below is from an image from my finder scope project using a 50 mm f1.4 lens while running at 25 fps. In order to calculate a quick and dirty instrumental-to-g transformation, I’ve taken stars with a g-band magnitude less than 10 and fit a line through them using a RANSAC regressor. What’s immediately apparent from the plot is that there is a pretty convincing trend through the data. Two actually. The first slopes upwards until a break is reached at about 10.8 Gaia g-band magnitude. After this point, the trend is flat. So, I’ll take the limiting magnitude as 10.8 for this setup. As this is an unfiltered system being compared to a known passband, there is a a fair bit of scatter and I’m not sure that we could do much better with the more precise synthetic injection method.

Two additional examples use data from a 50 cm f3 telescope equipped with an FLI Kepler 4040. The first is a 25 ms exposure while the second is a longer 10 s exposure. Although with the short exposure there are relatively few detections (less than 100 in total), there are still enough to define a trend through the data. In this case, the limiting magnitude is about M13. There is no flattening of the curve and there are no detections fainter than this limit.

With a longer exposure, there are naturally more detections and the trend is better-defined. In the figure below we see that there is actually a deviation from the dominant trend for detections below about M18.5. Yes, there are likely a couple of real detections that are fainter (down to M20) but there are also quite a few missed objects between M18.5 and M20.

Improvements can be made by simply counting the number of number of objects that should be made for a given magnitude within the field-of-view and comparing it to those that are actually found by SExtractor.

Summary

More to come…

The post How to find stellar magnitude limit – A bit of a hack first appeared on OBAFGKM.

The post How to find stellar magnitude limit – A bit of a hack appeared first on OBAFGKM.

]]>
https://www.obafgkm.ca/2019/12/05/how-to-find-stellar-magnitude-limit-a-bit-of-a-hack/feed/ 0 645
Cloud monitor MKII (now with POE!) https://www.obafgkm.ca/2019/12/02/cloud-monitor-mkii-now-with-poe/?utm_source=rss&utm_medium=rss&utm_campaign=cloud-monitor-mkii-now-with-poe https://www.obafgkm.ca/2019/12/02/cloud-monitor-mkii-now-with-poe/#respond Mon, 02 Dec 2019 16:09:13 +0000 http://orebits.ca/?p=85 After building the Mark I cloud monitor, I realized that I really wanted something with power-over-ethernet (POE). The cheap network adapters that I use typically have an HR911105 (or similar) RJ45 connector which is not compatible with POE. I’ve looked but have yet to find any of these adapters with POE, so I decided to hack my own together. In theory, it should be easy… What you need 2x MLX90614 thermopile sensors 1x Arduino Nano 1x W5500 ethernet adapter (I…

The post Cloud monitor MKII (now with POE!) first appeared on OBAFGKM.

The post Cloud monitor MKII (now with POE!) appeared first on OBAFGKM.

]]>
After building the Mark I cloud monitor, I realized that I really wanted something with power-over-ethernet (POE). The cheap network adapters that I use typically have an HR911105 (or similar) RJ45 connector which is not compatible with POE. I’ve looked but have yet to find any of these adapters with POE, so I decided to hack my own together. In theory, it should be easy…

What you need

  • 2x MLX90614 thermopile sensors
  • 1x Arduino Nano
  • 1x W5500 ethernet adapter (I use the tiny one for this project)
  • 1x IP camera cable
  • 1x IP camera POE module (optional if your cable puts out 12V)
  • 1x RJ45 connector
  • a bunch of jumpers
  • waterproof enclosure (I printed one)
  • Some way to mount it over the ground

If you want to print out my housing, you’ll also need the following…

  • A 3D printer (obviously) and weather-resistant filament (I use CPE/PETG)
  • Silicone
  • 1/4-20 nut
  • tripod ball mount

How to do it

Although, for the most part you’ll be following the instructions in my MKI post, there are a couple of steps that are different. Instead of exposing the RJ45 jack on the W5500 to the world, you’ll be running the IP camera cable through the housing and then crimping an RJ45 connector onto the the Tx and Rx wires of the cable. The 12V and ground wires will be soldered directly to the Nano on the input power pins 29 and 30 (GND and Vin). That is if you’re sure that the on-board voltage regulator is good for 12+ volts.

More to come…

The post Cloud monitor MKII (now with POE!) first appeared on OBAFGKM.

The post Cloud monitor MKII (now with POE!) appeared first on OBAFGKM.

]]>
https://www.obafgkm.ca/2019/12/02/cloud-monitor-mkii-now-with-poe/feed/ 0 644
Building a network-connected cloud sensor (MKI) https://www.obafgkm.ca/2019/11/30/building-a-network-connected-cloud-sensor/?utm_source=rss&utm_medium=rss&utm_campaign=building-a-network-connected-cloud-sensor https://www.obafgkm.ca/2019/11/30/building-a-network-connected-cloud-sensor/#respond Sat, 30 Nov 2019 00:56:41 +0000 http://orebits.ca/?p=40 I’ve wanted to build a cloud sensor for years now, but never actually found the time to do it. So, after seeing IR sensors for something on the order of $5 a piece, I ordered a couple and thought that I’d give it a shot. This has undoubtedly been done by lots of other people, but I thought that I’d share some simple instructions on how to build your own. The theory We could get in to the theory of blackbodies (maybe later), but it is sufficient to say that… Clouds are warmer than a clear sky. The infrared spectrum of an object can tells us something about its temperature. If you can measure the temperature of the sky, you should be able to say something about whether it’s cloud-free or not. To measure the temperature we need some sort of contact-less thermometer that can be pointed at the sky (and ground, more on that later). What you need 2x…

The post Building a network-connected cloud sensor (MKI) first appeared on OBAFGKM.

The post Building a network-connected cloud sensor (MKI) appeared first on OBAFGKM.

]]>
I’ve wanted to build a cloud sensor for years now, but never actually found the time to do it. So, after seeing IR sensors for something on the order of $5 a piece, I ordered a couple and thought that I’d give it a shot. This has undoubtedly been done by lots of other people, but I thought that I’d share some simple instructions on how to build your own.

The theory

We could get in to the theory of blackbodies (maybe later), but it is sufficient to say that…

  1. Clouds are warmer than a clear sky.
  2. The infrared spectrum of an object can tells us something about its temperature.
  3. If you can measure the temperature of the sky, you should be able to say something about whether it’s cloud-free or not.

To measure the temperature we need some sort of contact-less thermometer that can be pointed at the sky (and ground, more on that later).

What you need

  • 2x MLX90614 thermopile sensors
  • 1x Arduino Nano
  • 1x W5500 ethernet adapter
  • a bunch of jumpers
  • waterproof enclosure (I printed one)
  • Some way to mount it over the ground

If you want to print out my housing, you’ll also need the following…

  • A 3D printer (obviously) and weather-resistant filament (I use CPE/PETG)
  • Silicone
  • 1/4-20 nut
  • tripod ball mount

When you print, you’ll want to print with supports. This design has a cupped bottom and a rounded top to, hopefully, keep some of the rain out. I printed on an Utlimaker 3 using Innofil transparent PET filament with Ultimaker breakaway for support. It worked well, but I don’t really like the UM breakaway material with the Innofil filament. Cleanup was time consuming.

Putting it together

The MLX90614 sensors ship with a fixed i2c address. 0x5A, to be exact. You can have many i2c devices on the same bus, but each must have a different address. So, you need to figure out a way to change the address on one of the sensors. Fortunately, this is fairly easy to do.

  1. Make sure that the SparkFun MLX90614 library is installed
  2. Connect one (only one) sensor to your Arduino. SDA goes to pin A4, SCL goes to A5.
  3. Open the MLX90614_SetAddress example
  4. Make sure that oldAddress is set to 0x5A and newAddress is set to 0x5B
  5. Upload to your Arduino
  6. Open a serial monitor set to 9600 baud
  7. Follow the instructions
  8. Easy, peasy, lemon squeezy.

Now that there are two different i2c addresses, both devices can be hooked up to the Arduino. At this stage I test using the following code…

//*******************************************************************************
// Reads from two MLX90614 thermometer boards and prints the object temperatures
// and the difference between the two.                                          
// 
// Modified from the MLX90614_Serial_Demo found in the library at,
//https://github.com/sparkfun/SparkFun_MLX90614_Arduino_Library
//*******************************************************************************

#include Wire.h; // I2C library, required for MLX90614
#include SparkFunMLX90614.h; // SparkFunMLX90614 Arduino library

const byte IRAddress1 = 0x5A;
const byte IRAddress2 = 0x5B;

// Create 2 IRTherm objects
IRTherm therm1;
IRTherm therm2;

void setup() 
{
  Serial.begin(9600); // Initialize Serial to log output

  // Initialize the sensors and set the units
  therm1.begin(IRAddress1);
  therm1.setUnit(TEMP_C);
  therm2.begin(IRAddress2);
  therm2.setUnit(TEMP_C);
}

void loop() 
{
  // Call thermx.read() to read object and ambient temperatures from the sensor.
  if (therm1.read())
  {
    Serial.print("Object 1: "); Serial.print(therm1.object()); Serial.println("*C");
  }
  
  if (therm2.read())
  {
    Serial.print("Object 2: "); Serial.print(therm2.object()); Serial.println("*C");
    Serial.print("Temperature Difference: "); Serial.print(therm1.object()-therm2.object()); Serial.println("*C");
  }
  
  Serial.println();
  
  delay(2000);
}

//Hardware Hookup:
//MLX90614 ------------- Arduino
//  VDD ------------------ 3.3V
//  VSS ------------------ GND
//  SDA ------------------ SDA (A4)
//  SCL ------------------ SCL (A5)

Once the two sensors are working properly, the W5500 ethernet adapter can be connected and tested. The adapter is an SPI device and can be connected as follows…

W5500 Arduino
MOSI 11
MISO 12
SCLK 13
SS 10
RST 9
5V 5V
GND GND

When writing the code for the W5500 adapter you’ll need to keep in mind that you want to be using the ethernet2 library. Also, these modules typically don’t come with a fixed MAC address – you’ll be setting your own in the code, so you can get creative. For this, I’d recommend choosing a ‘proper’ locally administered MAC address instead of something completely random. By proper, I mean that the least-significant bit of the first octet of the address needs to be a 1. So your address needs to look something like…

x2:xx:xx:xx:xx:xx
x6:xx:xx:xx:xx:xx
xA:xx:xx:xx:xx:xx
xE:xx:xx:xx:xx:xx

where the x’s can be hex value you want. I chose BA:D1:DE:A5 to fill my first 4 octets and use the last two to number the devices on the network. A quick google search will yield lots of interesting hex ‘words’ that you can use if you want something memorable.

The code

I’ll be the first to admit that I’m not a master coder when it comes to C/C++. I get by on the Arduino thanks to the help of the ‘Googs’ and the examples that are often provided with libraries. This project is no different. Thanks to the SparkFun MLX90614 and Ethernet2 libraries, I was able to cobble something together that works reasonably well.

What I wanted to do was simply talk to the cloud monitor every once in a while to ask for current readings and record them in a local log file. As well as creating a log, I also wanted to plot the data at regular intervals and save a PNG that could be copied to a web-accessible place for viewing.

Insert code here…

Running it

Once everything is assembled and the silicone has dried, the cloud monitor can be connected to a computer or 5V power supply through the mini-USB connector. A network cable can be connected and then you should be able either navigate to its webpage or request packets UDP packets depending on what code you’ve uploaded to the Nano.

Some results

 

Some initial tests of the monitor show that it’s pretty good at detecting cloudy skies. The image above (several days of data) shows that, when it’s cloudy (seemingly all of the time right now), the temperature difference between the sky and ground sensor is only a few degrees. When it starts to clear up, however, the difference increases dramatically and can be up to 35 degrees during daytime clear skies. Although temperature differences of 20 degrees are a lot better than the cloudy sections, the skies are often still covered with a thin cloud.

Plotting a single day of data (below) we see the typical appearance of a large, ca. 30 degree, difference during the daytime when it’s clear and ca. 20 degree differences during night time clear periods.

Coming up

I don’t like having a separate power cable for this unit, so Mark II will have POE. It turns out that it’s a pretty simple mod that I wish that I had made the first time around.

The post Building a network-connected cloud sensor (MKI) first appeared on OBAFGKM.

The post Building a network-connected cloud sensor (MKI) appeared first on OBAFGKM.

]]>
https://www.obafgkm.ca/2019/11/30/building-a-network-connected-cloud-sensor/feed/ 0 40