OpenSensorData.net is a hosting service where you can store streams of data that come from any sensor or sensor networks. The use of OpenSensorData.net is free (like in free beer). And the software is Free (like in freedom), under the GPLv3. You can download the code and install it on your own servers (requirements: Apache/PHP/MySQL).
The first thing that is important to know is that we store all the data that you upload to OpenSensorData.net (that’s what it’s for) AND that we make all of this data freely available to any one under the Open Database License. The full text of the Open Database License can be found at http://opendatacommons.org/licenses/odbl/. Any rights in individual contents of the database are licensed under the Database Contents License whose text can be found http://opendatacommons.org/licenses/dbcl/. If you find it difficult to comply with this, then you should set up your own data servers.
The second important thing is that at any time you can ask us to remove ALL of the data associated with your account. We will make it possible for you to do this yourself in the future. For now, however, you have to send your request by email to the following address: someone AT opensensordata.net.
The source code of OpenSensorData.net is available under version 3 of the GNU General Public License. That means you can use, study, and modify it, and that you redistribute modified copies of the source code. The lastest version of the code is available here.
A datapoint consists of the time at which the measurement took place and its output value. For example, on Sunday, March 3rd 2013, at 11.55 am, the temperature sensor at my home measured 8.63 degrees celsius. The data point consists of the date (2013-03-10 11:55:00, for the precise format of the date, see details below), and its value (8.63).
A datastream is a series of datapoints. To add a datapoint to a datastream, you must first create the datastream. The minimum information you have to specify is a name, but a longer discription is welcome to. In addition, you can add information such as the geographical location of the sensor, and the timezone in which it is situated. All datastreams have a number that uniquely identifies it. To add a datapoint you must specify the datastream in which you want to insert it. You can pass along the datastream id, but there are other ways too (see “Upload your data”).
A group is a set of datastreams that you would like to keep together. Nearly all sensor boxes or data loggers will generate several datastreams and it obvious that you want to work with these datastreams together. Groups are flexible and it is also possible to create a group to bundle datastreams from different locations and from different users.
The creation of datastreams and groups is handled in the next section.
In most cases, you will not have to create groups and datastreams yourself. If the sensor box that user OpenSensorData is well designed, you should only provide the API key and a group name, and the device or software will create the group and its datastreams automagically.
In the short future, we will provide a online editor to manipulate groups and datastreams, however.
To create a groups and its datastreams, you must send a JSON description to the server over HTTP. Note that you must always specify the API kay in the X-OpenSensorData-Key header. The group definition contains a name and a description. A name can only be used once per account to label a group. So if I have a sensor box and it uses group name ’greenhouse’, if I install a second greenhouse I will have to choose another name.
The set of datagroups are coded in a JSON array, and for each you must provide its name and description. The datagroups will be automatically created for you.
The following example shows the complete HTTP request, with the headers, following by and empty line, and then the JSON description in the body.
A simple way to create the group, is to use telnet, connect to opensensordata.net on port 80, and then copy/paste the example above.
It is possible to add geolocation data to the datastream description. This allows the datastream to be positioned on a map. two extra fields are used, the longitude and latitude. You can specify a timezone. This can be useful for devices without a real-time clock. Lastly, you can specify a the unit in which the value is measured. For a complete list of currently supported units, see the list at the end of the documentation.
All the error handling in OpenSensorData.net is handled through the HTTP status codes. This makes it easier to communicate to the server in scripts. Status code 200 indicates that the request was handled successfully. Pretty much anything else will indicate an error. The body of the response will then contain a simple plain text error message.
If all goes well, the request above should have returned:
As you can see, the server returns the same JSON structure, but with an additional ID field for the group and the new datastreams. These IDs uniquely define the objects and can be used in other requests, to upload data, for example.
Variations of the PUT request can be used to modify the definition of an existing group. To update a group, or datastream, you must explicitely specify the IDs. In result of the following request, the name of the group will be changed from ’greenhouse’ to ’SmallGreenhouse’. Note that no definition of the datastreams is given. As a result, no changes to the set of datastreams of the group will be made. The server response will contain the complete description of the group in JSON, like above.
Had we given a description of the group’s datstreams, then these would have been updated, too. For example, you can remove all the datastreams from the group using the request below, in which the datastreams field is explicitely set to an empty array. The datastreams themselves will not be deleted. They will simply be dissociated from the group.
It is not necessary to create a group to create datastreams. Datastreams can be created directly using a PUT request similar to the one for groups. The URL to which you must send your request is http://opensensordata.net/datastreams/. Here’s an example:
The name of the datastream is unique for every user, i.e.. a user X can only create one datastream called Y. The response of the request above will return the JSON description of the new datastream, including its unique, global ID number. When the datastream is created, any follow-up PUT request will update the description of the datastream, except its ID.
You can upload a set of datapoints through the following URL: http://opensensordata.net/upload. The upload interface is designed to accept many datapoints that have been produced by a sensor device and that are uploaded through software, not by humans manually.
To upload data, you must send an HTTP POST request to http://opensensordata.net/upload, with all the datapoints in the body of the request. The format of the datapoints is the CSV file format (Comma Separated Values). This is a textual file format in which each line represents one datapoint. Each line should contain at least three fields: the ID of the datastream, the date, and the sensor value. All field values of the datapoint are separated by commas.
The following example makes this clear. The first five lines in the example are the HTTP request headers. You can pass the key of your account in the X-OpenSensorData-Key header.
The first column identifies the datastream. It is possible to mix datapoints of several datastreams together in the same request. The second column is the timestamp. OpenSensorData.net uses the ISO_8601 standard to encode the date and time. The last column is the value.
It is also possible to add geolocation data, for those of you who move around with the sensor device. The fourth column should contain the longitude, and the fifth the latitude:
The key of your account can be passed in the X-OpenSensorData-Key header. It is also possible to pass the key as a query string. The following example uploads a CSV file directly using the wget program:
The unique, numerical datastream identifier is the shortest way to specify the datastream. There is a second way to identify the datastream that is more human-readable. You can combine the account name, the group name, and the datastream name to identify the stream, as in the following example:
If the geolocation or the timezone is not specified in a datapoint, then the geolocation and timezone of the datastream will be used.
To obtain all the data of a datastream with ID [id], use the following URL: http://opensensordata.net/datastreams/[id].csv. We mentioned above that a datastream can also be identified using the account name and so on. This can be used here as well: http://opensensordata.net/datastreams/peter/greenhouse/temperature.csv.
If you want to select a specific time slice, you can extend the URL as follows: http://opensensordata.net/datastreams/[id]/[year]/[month]/[day]/[hour]/[minute]/[second].csv. It is not necessary to specify down to the second. Any intermediate range value will be exepted. For example, to get the data from March 2013, you can use: http://opensensordata.net/datastreams/peter/greenhouse/temperature/2013/03.csv.
If you use OpenSensorData.net for an application, you will probably have you own visualisation tools. Your app obtains the data through the URL explained above and displays the results in whatever way that makes sense. It is still handy to be able to inspect the data online at OpenSensorData.net. also so other people can view your data.
We currently use dygraph to make the figures. The URLs are quite similar to the ones above: http://opensensordata.net/viewdata/[id].csv. For example, http://opensensordata.net/viewdata/peter/greenhouse/temperature/2013/06.csv.
It is also possible to view all the datastreams in a group: http://opensensordata.net/viewgroup/peter/greenhouse/2013/06.csv.
Besides datastreams, you can also create photostreams to upload the output of image sensors. Photostreams are very similar to datastreams, except for uploading the data which must be done one photo at a time (see below).
The URL to define or modify a photostream description is http://opensensordata.net/photostreams/. Here’s an example:
The name of the photostream must be unique for every user, i.e.. a user X can only create one datastream called Y. The response of the request above will return the JSON description of the new photostream, including its unique, global ID number. When the photostream is created, any follow-up PUT request will update the description of the photostream, except its ID.
Like datastreams, photostreams can be created and modified as part of a group. The processing of the group description, looks for the ’photostreams’ field, which must be an array:
The JSON description of a particular photostream can be obtained using the URL http://opensensordata.net/photostreams/id.json. The ID is the unique numerical ID of the photostream, or the combination of the username, the name of a group, and the name of the photostream separated by slashes. For example, http://opensensordata.net/photostreams/peter/greenhouse/webcam.json, or http://opensensordata.net/photostreams/1.json.
For upload a JPEG photo, use a PUT or a POST request to the address http://opensensordata.net/photo/stream-id/timestamp.jpg. The stream-id is, like above, the numerical ID or the combination of user, group, and stream name. The timestamp is a concatenation of the date and time. For example, 20130511-170002.jpg. The year is coded in 4 digits, all the other fields use 2 digits.
The body of the request should contain the image data, without any additional information. This makes it easier for small clients to upload images. To push images from a shell script to the server, you can use, for example, the following command:
Currently only JPEG photos are accepted. We have also introduced some upload limits to reduce the exposure to software glitches or abuse of our service. The maximum image size is limited to 2MB and the maximum number of uploads that you can do per week per account is 100. If these limits are too constraint for your project, you should contact us and we will relax them.
You can retreive an image using the same URL as in the upload. For example, the image uploaded above is available at http://opensensordata.net/photo/1/20130524-154300.jpg. The server also makes a small (320x240) and a thumbnail (100x75) version of the image available. They have the “_s” and “_t” postfix, respectively: 20130524-154300_s.jpg for the small image and 20130524-154300_t.jpg for the thumbnail. The name 20130524-154300_o.jpg can also be used for the original image.
The list of images in a photostream can be retreived using http://opensensordata.net/photos/stream-id.json,txt. The first options returns a JSON array; the second the flat text with one photo URL per line.
The URL can contain a specific date, similar to getting the datapoints from a datastream. To get the list of photos from May 24th 2013, use http://opensensordata.net/photos/1/2013/05/24.txt.
For all requests, you should check the HTTP status code of the response. If the server returns ”HTTP/1.1 200 OK”, all went fine. Otherwise, the HTTP status class will give you idea of what went wrong. Most likely, one of the following three errors occured: unauthorized access, a bad request, or an internal server error. More details about the error can be found in the body of the response, which contains a plain text message, like in the example below:
Returns: datastream description, or HTTP error code
Returns: HTTP status code
Get the data of the datastream
ID: the datastream’s unique numerical ID, or the combination of username/groupname/datastreamname
Returns: the data
Get the datastream description
ID: datastream unique ID, or username/groupname/datastreamname
Returns: the datastream description, or HTTP error code
Get the group description
ID: the group’s unique numerical ID or username/groupname.
Returns: The new photostream description, or HTTP error code
Get the photostream description
with id: the photostream’s unique ID, or username/group-name/photostream-name
Returns: The photostream description, or HTTP error code
Upload a photo
with id: the photostream’s unique ID, or username/group-name/photostream-name
Returns: HTTP status code
Get a photo
URL: (orig version): http://opensensordata.net/photo/id/yearmonthday-hourminutesecond.jpg
or (orig version, explicite): http://opensensordata.net/photo/id/yearmonthday-hourminutesecond˙o.jpg
or (small version, 240x180): http://opensensordata.net/photo/id/yearmonthday-hourminutesecond˙s.jpg
or (thumbnail version, 100x75): http://opensensordata.net/photo/id/yearmonthday-hourminutesecond˙t.jpg
Returns: HTTP status code
Get the list of photos in a photostream
ID: the photostream’s unique numerical ID, or the combination of username/groupname/photostreamname
Format: JSON or text
Returns: JSON: An array of the photo descriptions; text: one photo URL per line
|Dimension||Short name||Long name||Comment|
|loudness||dB SPL||decibel sound pressure level|
|magnetic flux density||T||tesla|
|catalytic activity||kat||catalytic activity|
Copyright 2013, Sony Computer Science Laboratory Paris. This documentation is available under the GNU Free Documentation License.