Environmental Data Collector

My own litte Big Data collector

Project motivation

Somebody asked me to tell her a few words about my homebrew weather station. OK, here it is:

It isn't weather station in the literal meaning, the weather data rather is some kind of collateral. In the past, I was developing data loggers and power meters (of a highly sophisticated kind) for monitoring the power network. I had some spare time and the freedom to modify and abuse these things. So I started to add some features to these boxes, like acquisition and logging of other physical quantities.

Starting round about 2005 (I don't know the exact year), the first thing I built were some temperature sensors. One of the loggers had a temperature sensor input (RTD, PT100), that I expanded by an external multiplexer. Now I was able to log 5 temperatures instead of one. The logger was able to store quit a impressive amount of temperature data in its internal memory, but from time to time I had to manually read out that memory. Boring.

Somewhat later I expanded that logger by a handful of high precision sensors (SHT15, temperature and humidity). These use a digital protocol that wasn't supported at all by the concept of the data logger. Now things got somewhat more interesting. I hacked the hardware to convert some of the digital inputs to an appropriate interface to the sensors. The firmware (I *really* don't know how *this* could happen) was magically able to read out the sensors (hint: I was developing this data logger). The sensors were placed at various places inside and outside my home. These were in fact the first weather data that I collected.

About the same time I installed one of the power meters to monitor the power consumption of my home. This thing also had a versatile firmware. By chance, one of my tasks was to expand this firmware to support a local field bus. I needed some test targets, the temperature data logger did a fine job. As a collateral from now on I was able to read the temperature and humidity data into the power monitor, which has way more sophisticated data logging abilities. So I just connected the data logger to the power monitor to collect my weather data.

At some point, a defective rain gauge appeared on my desktop. It was part of a wireless weather station, but the wireless part of that thing was broken beyond repair. I used the still working mechanical part and a wired connection to the data logger to start collecting rain data.

So, now I'm collecting environmental data and cannot watch them? That's rather dissatisfying, isn't it? A nice friendly ARM did the job. I bought one of these kits (they're running linux on a ARM CPU and have a TFT screen), and wrote some software to display a choice of my temperature, power and whatever data. The result sits on my desktop, but is virtually available to me at any place with a decent internet connection, since I've ported its software to run on any linux or windows box.

The power monitor uses its internal memory to log the data, from time to time I have to read out that memory. Remember? Yes, that's stupid and boring. So I started to automate this process. A cron job running on a tiny linux box connects periodically to the power monitor and reads out the latest collected data. The data is stored in RRDs (Round Robin Database). This a really nice tool to collect data and create graphs from that data. See some results here: My public webcam page.

So what's my main motivation to do this? Just because I can! I like to see results of my work, and I like to see things working. I also like to take new challenges, mostly of electrical and technical kind, but not necessarily limited to this area. I like to keep up with the engineering progress of electronics, but I don't like things like smartphones and facebook. I rather prefer to keep my engineering skills alive and up to date than consuming internet and goods like the latest 3D 4k TV. I don't put too much effort into this work to make it look nice and shiny. But this thing is always changing and growing, at the moment I'm plannig to replace the veteran data loggers with recent hardware. Which would be homegrown, and I'd be able to publish schematics and source codes here.

I'm collecting lots of data with this machinery, most of the data not addressed to the public (things like power, gas and water usage, temperature of the living room and the kitchen sink...). The weather data are available as graphs and instantaneous values to view by humans, if someone asks me to, I might be able and willing to supply them in other formats.

You might notice the TI SensorTag within the pictures, I'm using three of them as wireless indoor sensors placed at places where wired connections would look ugly. I've modified the SensorTag firmware to act as need it for that purpose. I don't like wireless, and I don't like firmware like the TI SensorTag (it forces me to use a commercial toolchain, and its license doesn't allow me to publish the results). But sometimes you'll have to make compromises. In general I prefer to use free and open source tools, and in general I'm willing to open my work and its related results to the community.

Some pictures of the hardware

The rain gauge
A wireless sensor (TI sensortag)
One of the outdoor temperature / humidity sensors
One of the (visibly mounted, cellar only) indoor sensors
Electronics, part I (data aquisition and logging)
Electronics, part II (redundant power supply: solar, mains and battery, data gateway)
The large ambient light sensor, also used as a power supply
The small ambient light sensor
A blower used for venting the cellar rooms (controlled by sensor data)
This is where the white tube ends (with temperature sensor)
The raspi used as a GPS-disciplined NTP server (all the collected data shall have correct timestamps)
The desktop data display (includes an internet radio, guess my favourite radio station)
And its screen image

Back to Wunderkis.de (German language only) ... und ein Zaehlpixel hab ich auch :-)