Cam-era

My last posts discussed a little why the world desperately needed yet another weathers station, where I would place it and why and how the sensor package looked like. Now lets talk a little about data transmission.

Why

Given the position of the Sensor Package on roof of my house, a wireless solution was required.  Question is, which wireless system to use?  All the cool kids nowadays use WiFi modules like the ESP8266 to connect directly to the house WiFI network.  Its the obvious and the cleanest way of doing it as it also allows programming the Arduino over the wifi connection.  I did not go down that route for three reasons.

• Wifi is pretty bad up there and I would have to put a repeater near it (inside the house) to make sure I have a reliable connection.
• I really wanted a true mesh system that allows me to add sensors anywhere on the property, basically a backbone for home automation, alarms, chicken coop, hydroponics etc.  The more sensors (within reason), the denser the mesh, the more reliable the communication.
• I thought building a radio communication from “ground” up mostly as a challenge to myself.  Yes, this was the real reason.

In the end, I chose to use the Xbee standard from Digi since it is a very well documented flavor of the even better documented Zigbee (IEEE 802.15.4).  Xbee allows true mesh radio systems with relatively low programming, a lot of the network maintenance code is already build in.

As a recap on mesh networks, this page explains them very well. The advantage is that it allows good network expansions since the devices form a communication chain. As long as there is a coordinator, a remote device can send data across multiple “hops”.  The downside is that the data transmission rate is pretty low – a lot of bandwidth is burned just to establish the mesh itself.  Eventually, my house-network will have maybe 25 sensors and devices permanently powered that periodically send a short burst of data (weather stations) or battery operated end devices that only come online when they have something to send (door / window sensors, driveway sensors etc).

Lastly, mesh networks have a few interesting features, in that a single device attempts to communicate with many other devices and assesses which signal strength (RSSI) is the strongest.  For static sensors, this is not a big deal.  But in the back of my mind I had the idea of using this system to track small, mobile objects (=cats) and use the RSSI as a means to identify where they are.  To be honest, I wanted to build it because several intelligent and knowledgeable engineers told me that it is a bad idea and I thought it would be cool to prove them wrong. But that is a project for later.

The picture above shows three sensors (coordinator, weather station and a Laptop (blue, left) communicating with each other.  The signal strength for each path is shown on the arrows.

Back to the weather station.  I have one XBee configured as Router (inside the Sensor Package) and another as Coordinator connected to an Arduino which also doubles as a clock, display unit and general “hub”.   But while Arduinos excellent in interacting with the world, their single-thread processing and limited storage make crummy computer replacements.   Eventually (so the internet tells me) I should move the coordinator to a Rasperry PI but for the same effort, I probably just use an old laptop.  As the my coordinator, I ended up using a Arduino Mega since an Uno already ran out of storage.

How

To run an Xbee inside a mesh as described above, I need to structure the data into a package that the coordinator understands and I spent way too much time trying to figure this out by following well-meaning YouTubers.  In the end I bought the excellent book “Building Wireless Sensor Networks” by  Matthijs Kooijman which made understanding and implementing the network extremely easy.

Alright so we have the weather station Sensor Package on the roof sending data every 30 seconds to the coordinator.  The entire sketch is on Gitub.

void sendPacket() {

detachInterrupt(AnemInterruptPin); //stop interrupts while sending package //this is probably overkill but I dont like taking chances with interrupts during transmissison
detachInterrupt(RainInterruptPin); //stop interrupts while sending package

WindSpeed = (AnemCount / sendInterval * 1000) * WindFactor;
MaxGust = (MaxGustCount / gustInterval * 1000) * WindFactor;
MinGust = (MinGustCount / gustInterval * 1000) * WindFactor;

if (MinGustCount == 0 || MinGustCount == 1000) { //messy code. Cant reset the MinGust to 0 since it will always be lower than any reading. Setting it to some large value
MinGust = 0;
}

MaxGustCount = 0;
MinGustCount = 1000; //messy way of doing it

RainRate = (RainCount * RainFactor); // accummulated raind during the interval. The true mm/[time] is done elsewhere
AnemCount = 0;
RainCount = 0;

In this function, I detach the two interrupts since I am paranoid that an interrupt will disrupt the transmission sequence. It may not be necessary but I felt it was safer. Then the wind speed is calculated from the data sheet of the anemometer, gusts are calculated as the delta between the maximum and minimum wind speed of 5 second intervals within the 30 second send interval.

// Prepare the Zigbee Transmit Request API packet
ZBTxRequest txRequest;
txRequest.setAddress64(0x0000000000000000); //I assume this is broadcast.


// Allocate 40 payload bytes. 10 parameters = 8x4 bytes + 1 byte.
// Appending this needs appending on the Coordinator as well - shouldnt one be able to use the length?

AllocBuffer<41> packet;

// Packet type, temperature, humidity
packet.append<uint8_t>(1); //this is the additional byte.
packet.append<float>(dhtT); //each parameter has 4 bytes
packet.append<float>(dhtH);
packet.append<float>(bmpT);
packet.append<float>(p0);
packet.append<float>(WindSpeed);
packet.append<float>(RainRate);
packet.append<float>(windDirection);
packet.append<float>(MaxGust);
packet.append<float>(MinGust);
packet.append<float>(millis()); //place holder for battery status or the like. not relevant

txRequest.setPayload(packet.head, packet.len());

// And send it
xbee.send(txRequest);
attachInterrupt(0, AnemAcq, RISING);
attachInterrupt(1, RainAcq, CHANGE);

The data package for the Xbee transmission is assembled very closely to the examples in Kooijman’s book. I was concerned that the package was getting too long but this has not been an issue yet. The package consists of 1 byte + 4 bytes for each parameter that I send, so a total of 41 bytes is sent every 30 seconds. (The code comment is wrong, it should read 10 parameters = 10×4 bytes) It works very well and the library takes care of the heavy lifting (HEX codes etc). One improvement would be to calculate the package length automatically but since the structure doesn’t change, its not a big deal. Obviously, every new measurement type increases the package on the sending and the receiving end by 4 bytes.

Improvements for future

I mentioned before that Xbees are perfect for “beacon-based” transmissions, i.e. let the coordinator run the timer and requests the data from the router rather than have each route send based on their own time.  This keeps the channel much more controlled especially when multiple Routers or end devices are in the system and space needs to be made for non-scheduled transmissions (such as status changes, door openings etc).  I’ll do that next year when the next round of sensors is installed.

One additional feature I did not install yet is “store and forward”.  In principle, radio communication is always slightly messy, environmental effects affect signal strengths, power outages can occur and in general “stuff just happens”.  Especially longer transmissions with a single checksum may get garbled.  The Xbee system has some safeguards but if power is lost to the coordinator for a few minutes, data will be lost.  Implementing a system to store incoming data until its confirmed as sent is standard in many wireless systems and likely something that I will look into in the future as well.

My next project is to improve the receiver end (coordinator), at the moment its a naked Arduino with a LCD display and second Arduino hanging on it with a WiFi shield.  It works but its very messy

Alright.  After not just one but 2 (!) quasi scientific / philosophical posts about the concept of an entirely superfluous weather station (overkill), lets get into the actual meat of it.

Lets look at the hardware and sensors that need to live outside.  I decided that I needed temperature, humidity and barometric pressure as bare minimum.  I really wanted wind speed, wind direction and gust levels.  I read about other weather stations having UV monitoring but considering that I would have it underneath trees, I didn’t think there would be much worthwhile data.

So, as sensors, we have a DHT-22 (temp, humidity) and a BMP180 and a Sparkfun Wind speed and direction, gusts, rain gauge.   The DHT22 and the BMP180 are very well documented and supported, many, many examples exist of them being integrated into Arduino projects.  The Sparkfun wind speed and rain sensors are a little less well documented and functionally a little bit more complicated as well.  All sensors are connected to an Arduino Uno and – eventually – housed in a Home Depot electrical junction Box.  In the photo below, you see the assembly before attempting to tighten the cables and you also see the black antenna cable that connects to the XBee radio.  That cable by the way lives inside the junction box because I didn’t trust it to survive outside.  This is far from ideal but it does work.  I’ll write later about the entire wireless hook up.

A couple of comments about the box.  I realized that the weather station would have to be on the roof of the house and relatively difficult to reach so I put quite a bit of thought into the box itself.  It had to be as watertight as possible, resist burrowing critters, rust, leaves falling, heat up over +40 C and below -20 C.  A standard electric junction box from Home Depot does just that. I drilled 2 holes, one on the left for power supply and the wind/rain sensor cables.  A second hole allowed me to connect the DHT22 and the BMP180 and place them on the outside of the junction box.  That in turn means I can more or less seal off the main electronics compartment and keep the Arduino relatively safe.

To protect the dangling sensors, I took some left-over PVC piping and glued it to the box with standard plumbing glue .  I used little scraps of duct take to secure the pins to a couple of crudely-cut breakout boards since I didn’t want to solder the sensors themselves.  I did this mainly so that I can swap the sensors out if they are damaged for example. This connection is definitely a weak point and I am keeping an eye on it.  Not happy but it works for now.

Lastly, and not shown in the picture, I covered the bottom end of the PVC pipe with a layer of wire mesh.  You can buy the little patches of mesh at home depot as well, generally used to repair screen windows and doors.  Basically, where we live, if a critter can get in, it will and the screen should prevent this.  In the end, turn the entire assembly upside down and mount on the side of the chimney.

From the photo above, one can see the whole assembly.  The white PVC tube that contains the humidity, temperature and barometric pressure sensor points downward and air flows past it freely.  It is nearly always in the shade and about as easily accessible as anything can be on a roof.  The anemometer, wind vane and rain gauge are installed on an old VHF antenna pole.  Its not as high as I want but the whole assembly is rock-solid.  One issue may come up in the winter when I fire up the wood burning stove.  I don’t actually know how much it warms the bricks up but this nearly 3 floors above the fireplace and bricks are pretty good insulators and the sensors don’t even touch the brick themselves.  I don’t think its a big deal.  And if the sensor does get heated, its a sign that I am wasting energy, so that would be an interesting outcome.  I wonder what he wood smoke does to the anemometer and wind vane.  Probably nothing.

By the way: power.  Originally, I had intended to make the entire thing solar powered which of course causes quite a few design issues.  Low power, sleep mode, solar chargers, all of which I had planned until I went on the roof and removed the old VHF antenna off to make space.

Turns out, that antenna was quite a beast for its days and had a directional motor underneath that could be controlled from a dial in the bedroom.  The photo on the left shows the bottom assembly after unceremoniously sawed off the roof and thrown down.

Anyway, because of this antenna relic, it turned out that I had a 4-wire, decent gauge cable going directly from the bedroom to the chimney above it.  This means, no batteries, no solar, no fancy footwork on keeping the power low.  Burn, baby burn those mA, see if I care.

It also opened an interesting thought about low temperature management, in theory, I could add a small 12V heating pad inside the junction box and trigger it when things get too cold.  Honestly, I don’t know how cold Arduinos can get before they die but this is something I keep in mind this coming winter.

But mostly, these old cables kept the requirements really simple for the power management.  For future installs / chicken coops, vegetable gardening etc, I need to revisit the solar option but not today.

Next post, I’ll get into the Arduino per se and what it does (more so, why).

So, I wrote in my last post why the world urgently needed a new weather station and I set out to find a good location for the actual sensors.  This process was interesting in its own right since it revealed that we have quite a few microclimates on the property.

Below is a winter photo of the house with 3 (of many more) elements that affect the microclimate:

1. The star indicates a pretty dense tree cover, the majority of the house is shaded from the sun during the day by tall White Oak.  Especially for wind speed and direction, this matters:  Where possible measuring devices should be in a large open area away from possible interference such as building, trees etc. It is generally accepted that measurements are based on readings at 10 metres (33 feet) above ground and the distance between the anemometer and any obstruction is at least ten times the height of the obstruction! (Reference).  According to this, I would go at least 10 meters higher (!) than the canopy and while that would surely be a fun project, it likely need a building permit (and look pretty ugly).  More importantly, whats the point?  What relevance would the windspeed 30m above the ground level have for this project?
2. The arrow in the front points along a meadow toward and up to the house.  Generally, the sun heats the meadow first during sunrise and a significant warm breeze comes up the little hill and hits the front of the house, channels to the left and right, even on quiet days, there is quite some wind on the deck (left) and the carport (right).  By the way, in the photo, the photographer is pointing nearly exactly East.
3. In the circle is the chimney which I had earmarked as permanent position for the weather station,  I figured, the roof is the only place where I can guarantee that the house itself is not shading the weather station from the wind or sun.  Unfortunately, the canopy shields the sensor from the full wind, at least as long as there is foliage.  You can’t win, eh?

Ok, so one way or the other, readings from a single sensor hub are limited to the specific microclimate of the sensor which can be quite varied.  Just exactly how varied this could be is really interesting and maybe something I explore by making a dozen or so sensors and distribute them in various places.  But that is for later.  For now I chose the position just over the chimney as “good enough” to be indicative of the overall climate around the house.  Later I will write a little about “where on the chimney” and yes, it matters.

The process of identifying a good position was interesting since it forced me to think about the environment in context (meadow heats up -> creates higher pressure than in forest -> creates wind -> creates evaporative cooling -> …) but also made me think what I actually want out of the weather monitor.  In general, I don’t really need it to understand the weather around the house.  Looking out of the window is a perfectly fine method and even better is to step outside and feel it.  No, the goal was to develop a system that monitors changing conditions inside a specific environment, understand the reasons behind the changes and – later interact with the environment.

The next step is to actually build the sensor and put it in place.