Secure App Access to Raspberry Pi Camera

In this post we will show you how to set up a standard USB webcam with a Raspberry Pi (RPI) and the AppMyProduct service for easy access and high security: There are already many good guides out there that demonstrate how to setup a webcam with the RPI – but they all leave the reader with a combination of firewall hassle and very poor security: You will have to setup port forwarding in the WAN router to access the RPI and there is no encryption or access control. So a lot of hassle and risk of strangers peeking through your webcam.

With a few additional, simple steps as compared to these existing guides, you can eliminate the firewall hassle, add strong encryption and a robust access control mechanism – Nabto’s AppMyProduct to the rescue!

rpi-logitechc270

RPI + Logitech C270

Once you are done, you will be able to securely access your RPI Webcam from anywhere through the AppMyProduct Video App – without any firewall fiddling or messing with DynDNS and self-signed certificates. As you will see below, the apps are freely available from the app stores – and the source code is available so you can tweak yourself. It is a hybrid app, so you can make many interesting changes and additions by just knowing basic HTML.

What you need

  1. A Raspberry Pi board with a Raspbian image installed. We have tested with 2nd and 3rd generation boards, 1st generation will most likely also work like a charm. raspberry-pi-3-hero-1-1571x1080
  2. A camera connected to the PI, this guide assumes a USB webcam but you can use anything that works with the Motion software package, including Raspberry Pi camera modules.
  3. An iOS or Android device.
  4. An AppMyProduct account – create one for free on www.appmyproduct.com, no credit card necessary and you can add several devices completely free of charge for development, testing, home and educational use.

 

Step 1: Log on to the RPI

Log on to your RPI, either directly if you have it connected to a display with keyboard or through ssh from a computer on your network:

$ ssh pi@192.168.1.207
pi@192.168.1.207's password:
...
Last login: Sat Sep 9 09:14:12 2017 from mclappe2.home
pi@rpi:~ $

The default password for the pi user is raspberry.

 

Step 2: Install software

The following must be installed:

  • Motion – webcam software
  • CMake, Git and Ninja – to build the Nabto remote access software

Log on to the RPI as described above and perform the following steps:

$ sudo apt-get update
...
Fetched 9,707 kB in 22s (439 kB/s)
Reading package lists... Done
$ sudo apt-get install motion
...
$ sudo apt-get install git cmake ninja-build
...
Unpacking ninja-build (1.3.4-1.2) ...
Processing triggers for man-db (2.7.5-1~bpo8+1) ...
Setting up ninja-build (1.3.4-1.2) ...

If you already have a running webcam and just want to setup remote access, only lines 1 and 7 are necessary.

Step 3: Configure the webcam software

In this step we must edit two configuration files on the RPI. If you have a favorite editor and know your way around, just do the following:

  • set “daemon on” and “stream_localhost on” in /etc/motion/motion.conf
  • set “start_motion_daemon=yes” in /etc/default/motion

For a bit more detail:

Log on to the RPI as described above and start the nano editor in the console to enable the webcam software to run as a background service:

pi@rpi:~ $ sudo nano /etc/motion/motion.conf

The nano editor on the RPI. It is highly recommended to learn another editor like vi or emacs if you are going to edit a lot of files - but nano seems a popular choice in tutorials.Change “daemon off” to “daemon on” (the line with the cursor above).

Next, as an extra security measure, you can disable insecure remote access to the camera. You can do this by setting “stream_localhost on” in the same file (when we install the secure remote access software in a moment, it will access the camera from localhost):

Screen Shot 2017-09-09 at 19.29.17.pngThe tricky part is to exit this nano editor … press “control-x” (to exit), press “y” (to save) and press enter (to use the default suggested filename).

Next, we must start the webcam software when the RPI boots:

pi@rpi:~ $ sudo nano /etc/default/motion

Screen Shot 2017-09-09 at 19.22.02

Set “start_motion_daemon=yes” and exit as described above (control-x, “y”, enter).

Finally, start the webcam service:

pi@rpi:~ $ sudo service motion start

 

Step 4: Build the remote access software

Perform the following steps on the RPI (ie, log on to the device or continue in the shell from above).

First, retrieve the remote access source code from the github repo https://github.com/nabto/unabto.git:

$ mkdir git
$ cd git
$ git clone https://github.com/nabto/unabto.git
Cloning into 'unabto'...
remote: Counting objects: 5745, done.
remote: Total 5745 (delta 0), reused 0 (delta 0), pack-reused 5745
Receiving objects: 100% (5745/5745), 2.63 MiB | 1.55 MiB/s, done.
Resolving deltas: 100% (3388/3388), done.

Next,  run cmake to prepare the build of the uNabto tunnel service. The key step in line 4 below is quite long, so scroll a bit to see it all or look below where we have zoomed in on it:

$ cd unabto/apps/tunnel
$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Release -DUNABTO_CRYPTO_MODULE=openssl_armv4 -DUNABTO_RANDOM_MODULE=openssl_armv4 -GNinja ..
-- The C compiler identification is GNU 4.9.2
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc
...
-- Generating done
-- Build files have been written to: /home/pi/git/unabto/apps/tunnel/build

The cmake invocation in step 4 looks as follows:

cmake -DCMAKE_BUILD_TYPE=Release \
-DUNABTO_CRYPTO_MODULE=openssl_armv4 \
  -DUNABTO_RANDOM_MODULE=openssl_armv4 -GNinja ..

Just FYI – the Ninja tool used is an alternative to standard make tool; it builds much faster on the RPI than make, likely because Ninja has optimized filesystem access on the quite slow RPI.

Now you can build the tunnel:

pi@rpi:.../build $ ninja
[77/77] Linking C executable unabto_tunnel

Check you can execute the unabto_tunnel binary (./unabto_tunnel) and copy the resulting binary into /usr/bin:

pi@rpi:.../build $ sudo cp unabto_tunnel /usr/bin

 

Step 5: Create an AppMyProduct device

For this step, you need an AppMyProduct account – go to https://www.appmyproduct.com and sign up if you haven’t done so already.

Next, create a product – you don’t have to tick the “Free product” checkbox, you get a few regular licenses for free with your account (in this way you don’t have to see ads for the first devices you use).

Screen Shot 2017-09-09 at 20.29.24

Next, generate a license by clicking the “Generate license” button and enter 1 as quantity:

Screen Shot 2017-09-09 at 20.36.18

You now have everything you need to run the tunnel on the device – the device id and the key revealed when clicking “Show License Key” are used in the next step:

Screen Shot 2017-09-09 at 20.38.32

Step 6: Configure the tunnel

Log on to the RPI or continue your session above. Edit the file ./git/unabto/apps/tunnel/scripts/unabto_tunnel.conf:

pi@rpi:~ $ cd git/unabto/apps/tunnel/scripts
pi@rpi:.../scripts $ nano unabto_tunnel.conf

Replace the device id and key fields with what you retrieved from the AppMyProduct portal in step 5:

Screen Shot 2017-09-09 at 22.01.07.png

The port number 8081 matches the default configuration of the Motion webcam service, change this if you use a different port number.

Install the startup script and config file from the scripts directory:

$ sudo cp unabto_tunnel.conf /etc
$ sudo cp unabto_tunnel_initd /etc/init.d/unabto_tunnel
$ sudo chmod +x /etc/init.d/unabto_tunnel
$ sudo update-rc.d unabto_tunnel defaults
$ sudo /etc/init.d/unabto_tunnel start

Now everything should be working like a charm! Before actually firing up the app and enjoying remote access to your webcam, let’s check everything looks ok:

$ ps auwwx | grep unabto_tunnel | grep -v grep
pi  16300 0.1 0.2 18428 1932 ? 21:00 0:00 /usr/bin/unabto_tunnel /usr/bin/unabto_tunnel -d vtquiht9.xmnqf.appmyproduct.com -s -k

If nothing is seen as output of the ps command, go back to step 4 and make sure you can execute the binary after building it – and that you copied it into /usr/bin. If this does not help, write a comment below with your error message.

Also check the webcam service is running fine:

pi@rpi:.../scripts $ ps auwwx | grep motion | grep -v grep
motion 15762 1.4 1.1 53716 10020 ?    Sl  20:31  1:13 /usr/bin/motion

If nothing is seen here, make sure everything was installed correctly in step 2. Also, check Motion’s FAQ.

Step 7: Access the webcam

Visit Apple’s App Store or Google Play to download the AppMyProduct Video app. Start the app on the same network as the RPI to discover the device by tapping “Add new” and pair on the subsequent screens:

After pairing, you can see the actual webcam feed from anywhere – your communication with the camera is encrypted and only paired clients are allowed access. You can change settings for the device to control remote access:

On the security settings page, you should disable “Open for pairing” when you don’t want to allow further clients access to the camera. In the Access Control List you can see all the paired clients, ie who has access. You can delete the individual users with access by tapping the element.

Note that the “player” is extremely simple – it can only be used to show the default MJPG stream from the webcam. In a later blog post in this series we will add support for more advanced players for better video quality and also for adding sound.

You can download the source code for the player app from the ionic-starter-nabto-video github repo – it is a hybrid app you can edit by knowing just basic HTML. We will get back to this in a later post, demonstrating how to add new features to the webcam app.

Further Reading on Security

Some of you have asked for a bit of elaboration on how access control works in more detail and how we make sure only the intended users can access the device. It is a bit out of the scope to go into detail with in this post – but the AppMyProduct Video app and the tunnel software installed on the RPI use the Nabto platform under the hood for communication.

The platform provides a few different ways to control access, in this project we use the “Paired Public Key Authentication” approach outlined in section 8.2 of TEN036 “Security in Nabto Solutions”. Basically this ensures that only clients that possess the private key of an RSA keypair where the public key has been installed on the RPI is allowed to access the camera. In addition to the introduction above, you can read more about it in the Fingerprint ACL module documentation in the uNabto SDK.

For a more general introduction to security in the Nabto platform, the remaining sections of TEN036 “Security in Nabto Solutions are a good read – and also, we have a short introduction in this blog post.

 

Secure P2P remote control of ESP32 WiFi FreeRTOS based Module with AppMyProduct

The ESP32 is a powerful WiFi module from the creators of the successful ESP8266 low-cost WiFi module, Espressif. The creators has released the ESP-IDF (Iot Development Framework) which is a FreeRTOS based core for the module that you can plug your own code and modules into.

The ESP32 module is a dual protocol hybrid with WiFi + Bluetooth + Ethernet with a 240Mhz dual core 600DMIPS CPU, 520KB RAM and 16MB flash, yet it uses very low power. All in all a much more powerful module as to what is needed for a uNabto integration – but this just leaves more space to the application.

Lots of different boards are out there, Espressif has released some, but we settled on the evaluation board from Olimex (ESP32-EVB) which has a good formfactor, good available interfaces, and even contains two 250VAC relays so it can be used for real applications out of the box (for example to remotely turn an appliance on/off). Also the hardware is open source, so you can download the pcb files and adjust it to match your own application.

The github repository for our project located here:

https://github.com/nabto/unabto-esp32

Experienced ESP32 developers might just need to read the readme file. But if you haven’t worked with ESP32, we advise you to read further on.

The Olimex ESP32-EVB Board and what you need

You can buy the board here: https://www.olimex.com/Products/IoT/ESP32-EVB/open-source-hardware

FEATURES

  • ESP32-WROOM32 module
  • Built-in programmer for Arduino and ESP-IDF
  • WiFi, BLE connectivity
  • Ethernet 100Mb interface
  • MicroSD card
  • Two relays 10A/250VAC with LED status
  • CAN interface
  • IR received and transmitter from 5 meter distance
  • LiPo charger for stand alone operation during power breaks with 4 status LEDs
  • UEXT connector to connect UEXT modules
  • GPIO 40 pin connector with all ESP32 ports
  • Dimensions: (75 x 75)mm ~ (2.95 x 2.95)

The board can be powered using an USB cable, so that is the only extra thing you need extra and you’re up and running. The USB is also the programming interface via a USB to serial builtin adapter.

What software you’ll need

The ESP32 contains two cores. An Arduino core and an ESP-IDF (IoT Development Framework) core. We have done many Arduino integrations so our idea was to do this specific integration via the ESP-IDF. So start by following the setup guide here:

https://github.com/espressif/esp-idf#setting-up-esp-idf

Start by installing the toolchain for your operating system.

 

How the Nabto platform works

So far, so good. But how does the Nabto platform actually work? The drawing below gives a brief overview. Your ESP32 module represents the Device running the uNabto server. As soon as it connects to the internet it identifies itself at the Nabto Basestation, using its unique ID. If a Client wants to connect to the ESP32 module, a connect request with the ID is sent to the Basestation, and a direct connection to the device is established. A client can be a native app or use an abstraction framework like our Heat Control Ionic starter app used in this demo.

nabto-platform-basics

The above description is from helicopter perspective. The basestation is actually not a single instance but multiple clusters of basestations located in different datacenters around the world located based on the ip-geo-location of the device. The basestations have access to a distributed database telling potential clients that want to connect to a device, which basestation the client need to communicate with. Basestations are actually a manager-node that manages multiple worker-nodes. But to you as a developer all this is hidden, you just need to put the device-id and key on the device (if it’s a new system integrate to the Nabto integration adapter) and you have a world wide accessible device.

Get more information on the AppMyProduct/Nabto platform and the Client/Device SDKs on portal.appmyproduct.com or nabto.com.

The uNabto Platform Adapter integration

The uNabto server is divided into two layers:

  • The actual uNabto framework (uNabto SDK)
  • The uNabto Platform Adapter abstraction layer between the framework and the Native Platform

unabto-platform-adapter

Hence, we only need to implement the uNabto Platform Adapter, in order to port the uNabto device code to the ESP32 module.

Implementing the Platform Adapter

The Platform Adapter acts as link between the generic uNabto framework and the ESP-IDF core. The adapter is divided into single files, as suggested in the Nabto documentation (TEN023 Nabto device SDK guide, section 12), and the files can be found in the “porting” directory:

  • unabto_config.h: Basic uNabto configuration
  • unabto_platform_types.h: Define all necessary uNabto types
  • unabto_platform.h: Platform specific ad-hoc functions
  • network_adapter.c: Init, close, read and write functionality for network data
  • time_adapter.c: Time functions
  • dns_adapter.c: DNS resolving
  • random_adapter.c: Random generator

If you are interested in how the platform adapter is implemented in detail, check the adapter files in the src directory of the library on GitHub. Since the ESP-IDF is based upon BSD sockets the integration is fairly easy. The logging adapter of uNabto has been tied to the “printf” command that writes to the serial interface.

 

Using the code

Just follow the instructions in the readme file of the github directory.

The github repository can be found here: github.com/nabto/unabto-esp32

You just need to clone the github repository:

~$ git clone --recursive https://github.com/nabto/unabto-esp32.git
Cloning into 'unabto-esp32'...
remote: Counting objects: 80, done.
remote: Compressing objects: 100% (57/57), done.
remote: Total 80 (delta 30), reused 60 (delta 17), pack-reused 0
Unpacking objects: 100% (80/80), done.
Submodule 'components/unabto' (git@github.com:nabto/unabto.git) registered for path 'components/unabto'
Cloning into '/Users/carstengregersen/unabto-esp32/components/unabto'...
Submodule path 'components/unabto': checked out 'e60857eb0effea22f418f972c7bfec6153595f20'

 

Enter into the unabto-esp32 directory and type:

cd unabto-esp32/
make menuconfig

You will enter into the ESP-IDF configuration editor.

Remember to adjust the serial flasher to match the configuration of the specific USB-serial port driver you installed!!!

After this go into the “Custom configuration” menu:

 

Screen Shot 2017-08-18 at 15.23.56.png

You should now enter the WiFi access point SSID and password that the module should locally attach to. Also enter the Nabto-Id and key that you retrieve from the AppMyProduct portal (Signup -> Specify a product (f.x. mytestproduct) -> Generate Licenses -> Retrieve key).

Screen Shot 2017-08-18 at 15.26.35.png

After this, compile the project (make), flash the firmware (make flash) and lastly monitor the log (make monitor).

make
make flash
make monitor

You should see something like

0:00:00:114 main.c(172) Connecting to nabtotest1
00:00:00:117 main.c(74) Main task: waiting for connection to the wifi network...
00:00:02:486 main.c(76) connected!
00:00:02:487 main.c(82) IP Address: 192.168.0.103
00:00:02:490 main.c(83) Subnet mask: 255.255.255.0
00:00:02:495 main.c(84) Gateway: 192.168.0.1
00:00:02:499 unabto_common_main.c(121) Device id: 'uqhtktt3.smtth.appmyproduct.com'
00:00:02:506 unabto_common_main.c(122) Program Release 4.1.1
00:00:02:513 unabto_app_adapter.c(690) Application event framework using SYNC model
00:00:02:519 unabto_context.c(55) SECURE ATTACH: 1, DATA: 1
00:00:02:524 unabto_context.c(63) NONCE_SIZE: 32, CLEAR_TEXT: 0
00:00:02:531 unabto_common_main.c(200) Nabto was successfully initialized
00:00:02:537 unabto_application.c(71) In demo_init
00:00:02:541 unabto_application.c(92) Before fp_mem_init
00:00:02:546 unabto_application.c(95) Before acl_ae_init
00:00:02:555 unabto_context.c(55) SECURE ATTACH: 1, DATA: 1
00:00:02:556 unabto_context.c(63) NONCE_SIZE: 32, CLEAR_TEXT:
00:00:02:565 unabto_attach.c(804) State change from IDLE to WAIT_DNS
00:00:02:568 unabto_attach.c(805) Resolving DNS for uqhtktt3.smtth.appmyproduct.com
00:00:02:585 unabto_attach.c(818) DNS error (returned by application)
00:00:02:587 unabto_attach.c(819) State change from WAIT_DNS to IDLE
00:00:04:595 unabto_context.c(55) SECURE ATTACH: 1, DATA: 1
00:00:04:596 unabto_context.c(63) NONCE_SIZE: 32, CLEAR_TEXT: 0
00:00:04:599 unabto_attach.c(804) State change from IDLE to WAIT_DNS
00:00:04:602 unabto_attach.c(805) Resolving DNS for uqhtktt3.smtth.appmyproduct.com
00:00:04:615 unabto_attach.c(824) Resolved DNS for uqhtktt3.smtth.appmyproduct.com to:
00:00:04:617 unabto_attach.c(830) <span class="Apple-converted-space">  </span>Controller ip: 52.31.229.62
00:00:04:622 unabto_attach.c(836) State change from WAIT_DNS to WAIT_BS
00:00:04:686 unabto_attach.c(487) State change from WAIT_BS to WAIT_GSP
00:00:04:686 unabto_attach.c(488) GSP address: 54.72.234.97:5565
00:00:04:695 unabto_attach.c(275) ########U_INVITE with LARGE nonce sent, version: - URL: -
00:00:04:750 unabto_attach.c(591) State change from WAIT_GSP to ATTACHED

The last line is indicating success:

State change from WAIT_GSP to ATTACHED

This tells that the device now is attached to a GSP (Group ServerPeer – which could be considered a basestation worker node).

Now download the Heatcontrol demo app. Pair it with the ESP32-EVB board and now you can remote access the board. NOTE: The ACL list is currently stored in RAM not stored in EEPROM, i.e. a reboot will reset the Access Control List.

Please contact our support@nabto.com if you have any questions or need any help with integration.

amp-heat-app-screenshot-control_framed

 

 

CC3200 Two-Way Audio with Android

Texas Instrument’s CC3200 is a single-chip microcontroller unit with built-in Wi-Fi connectivity, created for the Internet of Things. It can run the FreeRTOS operating system and provides a hardware encryption engine. Sounds interesting? It did to us! So we created a demo that is capable of streaming two-way audio between the CC3200 and an Android App using our uNabto framework. Why do you need Nabto for this? Because it solves all the router and firewall hassle for you: all you need to connect to the device is a unique Device ID!

cc3200-audio-board

What you need

Just hook up the two boards as described in the Audio BoosterPack User Guide provided by TI.

How the Nabto platform works

How does the Nabto platform work exactly? The drawing below gives a brief overview. Your CC3200 represents the Device running the uNabto server. As soon as it connects to the internet it identifies itself at the Nabto Basestation, using its unique ID. If a Client (i.e. the Android App in this demo) wants to connect to the CC3200, a connect request with the ID is sent to the Basestation, and a direct connection to the device is established.

nabto-platform-basics

Get more information on the Nabto/AppMyProduct platform and the Client/Device SDKs at appmyproduct.com.

The uNabto Platform Adapter:

The uNabto Platform Adapter is a small component that abstracts the Native Platforms network and time functionality. The Platform Adapter is part of the uNabto server.

The uNabto server is divided into two layers:

  • The uNabto framework (uNabto SDK)
  • The uNabto Platform Adapter abstraction layer between the framework and the Native Platform

unabto-platform-adapter

To run the uNabto server on the CC3200, we only need to implement the uNabto Platform Adapter (for details see the TEN023 Nabto device SDK guide). The CC3200 adapter is divided into the following files:

  • unabto_config.h: Basic uNabto configuration
  • unabto_platform_types.h: Define all necessary uNabto types
  • unabto_platform.h: Platform specific ad-hoc functions
  • unabto_adapter_network.c: Init, close, read and write functionality for network data
  • unabto_adapter_time.c: Time functions
  • unabto_adapter_dns.c: DNS resolving
  • unabto_adapter_random.c: Random generator
  • unabto_adapter_crypto.c: CC3200 hardware encryption

If you are interested in how the platform adapter is implemented in detail, check the adapter files on GitHub.

The Device Application

Info: The following describes the audio demo application (‘audio’ branch). A more generic and straightforward example of using streaming to echo data is maintained in the ‘master’ branch of our CC3200 GitHub repository.

The uNabto server is running in its own task implemented in unabto_task.c. After waiting for the network connection being established in another task, the uNabto server is initialized with basic settings as well as the unique Device ID and pre-shared encryption key from portal.appmyproduct.com.  Then, the server continuously handles incoming network events and checks for available recorded audio to send to the client.

void UNabto(void* pvParameters) {
    // device id and key from portal.appmyproduct.com
    const char* nabtoId = "<DEVICE ID>";
    const char* presharedKey = "<KEY>";

    // Initialize uNabto
    nabto_main_setup* nms = unabto_init_context();
    nms->ipAddress = g_uiIpAddress;
    nms->id = nabtoId;
    nms->secureAttach = 1;
    nms->secureData = 1;
    nms->cryptoSuite = CRYPT_W_AES_CBC_HMAC_SHA256;

    const char* p;
    unsigned char* up;
    for (p = presharedKey, up = nms->presharedKey; *p; p += 2, ++up) {
        *up = hctoi(p[0]) * 16 + hctoi(p[1]);  // hex string to byte array
    }

    while ((!IS_CONNECTED(g_ulStatus)) || (!IS_IP_ACQUIRED(g_ulStatus))) {
        osi_Sleep(500);
    }

    srand(xTaskGetTickCount());

    stream_audio_init();
    unabto_init();

    while (true) {
        wait_event();
        stream_audio_write();
    }
}

The actual audio streaming code is in stream_audio.c. The unabto_stream_event() function handles all incoming streaming events. Once a Nabto stream connection is established, the device is waiting for a stream command. In this case, there is only the command “audio”. If an unknown command is received, the device returns “-” and closes the stream. If “audio” is received from the client an acknowledgment is returned (“+”) and the actual audio streaming is started. Incoming audio data from the client is processed in lines 125-137:

    if (my_audio_stream.state == STREAM_STATE_STREAMING) {
		const uint8_t* buf;
		unabto_stream_hint hint;
		size_t readLength = unabto_stream_read(stream, &buf, &hint);
		if (readLength > 0) {
			adpcm_decode(pPlayBuffer, buf, readLength);
			if (!unabto_stream_ack(stream, buf, readLength, &hint)) {
				my_audio_stream.state = STREAM_STATE_CLOSING;
			}
		} else {
			if (hint != UNABTO_STREAM_HINT_OK) {
				my_audio_stream.state = STREAM_STATE_CLOSING;
			}
		}
}

The data is read from the stream, decoded, and written to the circular play buffer. Processed data has to be acknowledged with unabto_stream_ack(). For the audio encoding ADPCM is used (for implementation details see adpcm_audio.c). It compresses the two 16-bit stereo samples to one single byte. At 16000 samples per second, this results in a transfer bitrate of 16 kbit/s which is no problem for the CC3200, even for encrypted remote connections. If you want to have a more advanced codec like for example Speex, just replace the encoding and decoding functions.

As mentioned before, the uNabto task continuously checks for available recorded audio by calling the stream_audio_write() function. If new encoded audio is available and a stream is established, the data is sent.

void stream_audio_write() {
	if (my_audio_stream.state != STREAM_STATE_STREAMING) {
		return;
	}
	size_t encodedLen = adpcm_encode(pRecordBuffer, encodedBuf, sizeof(encodedBuf));
	if (encodedLen == 0) {
		return;
	} else {
		UNABTO_ASSERT(encodedLen == sizeof(encodedBuf));
	}

	unabto_stream_hint hint;
	size_t writeLength =
			unabto_stream_write(my_audio_stream.stream, encodedBuf, sizeof(encodedBuf), &hint);

	if (writeLength <= 0 && hint != UNABTO_STREAM_HINT_OK) {
		my_audio_stream.state = STREAM_STATE_CLOSING;
	}

	UpdateReadPtr(pRecordBuffer, writeLength * ENCODING_RATIO);
}

Using the CC3200 Code

You can get the whole CC3200 code including the described platform adapter and the audio streaming device application from the audio branch of our public CC3200 GitHub repository. Simply follow the instructions in the README to set everything up.

The Android Client

The Android audio streaming client is also published on GitHub. It uses our android client SDK available on JCenter (for source code see GitHub repository). It is included in the build.gradle with one single line:

compile ‘com.nabto.android:nabto-api:1.0.1’
}

The main App logic is implemented in MainActiviy.java. Once the unique Device ID is entered in the UI and the OPEN AUDIO STREAM button is pressed, a thread is started to establish the stream connection and send the “audio” command. Then, a recording + sending and a receiving + playing thread are started until the connection breaks or the stream is closed by the user.

android-audio-client

To run the App on your Android device, follow the README instructions in the repository. If you enter the Device ID, you should be able to establish a stream connection and hear the microphone (and line-in) input of the opposite device.

Remote control your Arduino MEGA and Wiznet Ethernet with AppMyProduct

Summary:

With this article you will learn how to create a firmware and an app to securely Internet remote control (from inside AND outside your home network) an Arduino MEGA board with a Ethernet Shield. The IoT communication infrastructure used for this is AppMyProduct.com a Software as a Service, Pay as you Go (with free demo licenses) version of the Nabto.com Enterprise platform which makes it easy to create remote control IoT systems for all types of microcontrollers and remote control clients (apps, tablets, pc’s etc.). The app and firmware is a simple Heating control demonstration which easily can be altered to your own need.

What you need:

  • Arduino MEGA2560 board
  • Wiznet W5100 Ethernet Shield version 1

Arduino MEGA:

(If you know all about Arduino, please scroll down to “About uNabto AppMyProduct” section)

The Arduino Mega 2560 is a microcontroller board based on the ATmega2560. It has 54 digital input/output pins (of which 14 can be used as PWM outputs), 16 analog inputs, 4 UARTs (hardware serial ports), a 16 MHz crystal oscillator, a USB connection, a power jack, an ICSP header, and a reset button. It contains everything needed to support the microcontroller; simply connect it to a computer with a USB cable or power it with an AC-to-DC adapter or battery to get started.

The Arduino Mega is the addition to the Arduino family. This board is physically larger than all the other Arduino boards and offers significantly more digital and analog pins. The MEGA uses a different processor allowing greater program size and more. The Mega2560 differs from all preceding boards in that it does not use the FTDI USB-to-serial driver chip. Instead, it features the ATmega16U2 programmed as a USB-to-serial converter. The Mega has four hardware serial ports, which means maximum speed if you need a second or third (or fourth) port.

ArduinoMega

Technical Specifications:

1. Microcontroller: ATmega1280 or 2560

2. Operating Voltage: 5V

3. Input Voltage(recommended): 7-12V

4. Input Voltage(limits): 6-20V

5. Digital I/O Pins: 54 (of which 14 provide PWM output)

6. Analog Input Pins: 16

7. DC Current per I/O Pin: 40 mA

8. DC Current for 3.3V Pin: 50 mA

9. Flash Memory: 128KB or 256KB

10. SRAM: 8 KB

11. EEPROM: 4 KB

12. Clock Speed: 16 MHz

Wiznet W5100 Ethernet Shield:

The Arduino Ethernet Shield connects your Arduino device to the internet. Just plug this module onto your Arduino board, connect it to your network with an RJ45 cable. As always with Arduino, every element of the platform – hardware, software and documentation – is freely available and open-source.

Specifications:

  •  Operating voltage 5V (supplied from the Arduino Board)
  •  Ethernet Controller: W5100 with internal 16K buffer
  •  Connection speed: 10/100Mb
  •  Connection with Arduino on SPI port

Description:

The Arduino Ethernet Shield allows an Arduino board to connect to the internet. It is based on the Wiznet W5100 Ethernet chip (datasheet). The Wiznet W5100 provides a network (IP) stack capable of both TCP and UDP.

The Ethernet Shield has a standard RJ-45 connection, with an integrated line transformer and Power over Ethernet enabled.

The shield also includes a reset controller, to ensure that the W5100 Ethernet module is properly reset on power-up. Previous revisions of the shield were not compatible with the Mega and need to be manually reset after power-up.
The current shield has a Power over Ethernet (PoE) module designed to extract power from a conventional twisted pair Category 5 Ethernet cable.

W5100-EthernetShield

Technical Specification:

  • IEEE802.3af compliant
  •  Low output ripple and noise (100mVpp)
  •  Input voltage range 36V to 57V
  •  Overload and short-circuit protection
  •  9V Output
  •  High efficiency DC/DC converter: type 75% @ 50% load
  •  1500V isolation (input to output)

Interfacing Arduino MEGA with Ethernet Shield:

Ethernet Shield Arduino board connects to a LAN or the Internet. Installation is very simple. Plug the Ethernet shield connectors in the expansion card connectors of Arduino and then connect the Ethernet cable to the RJ45 connector slot. In the image below you can see the Arduino Mega with an installed expansion board Ethernet Shield.

ArduinoMega-EthernetShield

About uNabto AppMyProduct:

AppMyProduct is an IoT application platform that helps you to
1. Quickly develop high-quality client side apps using SDKs and/or template apps
2. Customize the device side application using the provided demo application.
3. The uNabto framework and the client APIs can be downloaded using the below link.
https://www.nabto.com/downloads.html

uNabto Framework:

The drawing below gives a brief overview of how the Nabto platform actually work. The Device represents the Arduino Mega and the uNabto server (uNabto SDK and Device specific Platform adapters) is running on the device. As soon as device connects to the internet it identifies itself at the Nabto Basestation, using its unique ID which is already registered in AppMyProduct portal. If a Client wants to connect to the device, a connect request with the device ID is sent to the Basestation, and a direct connection to the device is established after verifying the identity of client. A client can be a native mobile app or an abstraction framework like our Heat Control Ionic starter app used in this demo

nabto-platform-basics

Get more information on the AppMyProduct platform and the Client/Device SDKs at appmyproduct.com.

The uNabto Platform Adapter:

The uNabto Platform Adapter is a small component that abstracts the Native Platforms network and time functionality. The Platform Adapter is part of the uNabto server.

The uNabto server is divided into two layers:

  •  The uNabto framework (uNabto SDK)
  • The uNabto Platform Adapter abstraction layer between the framework and the Native Platform

unabto-platform-adapter

The device specific uNabto server is readily available in AppMyProduct portal to port it to the Arduino module.

Implementing the Platform Adapter:

The Platform Adapter acts as link between the generic uNabto framework and the Arduino platform. The adapter is divided into single files, as suggested in the Nabto documentation (TEN023 Nabto device SDK guide, section 12):

  • unabto_config.h: Basic uNabto configuration
  •  unabto_platform_types.h: Define all necessary uNabto types
  •  unabto_platform.h: Platform specific ad-hoc functions
  •  time_adapter.cpp: Time functions
  •  random_adapter.cpp: Random generator functions
  •  dns_adapter.cpp: DNS resolver functions
  •  network_adapter.cpp: Communication functions
  •  log.cpp: Logging

Heat Pump Demo:

The Heat Pump demo showcases how to use the Nabto request response model on the Atmel AVR platform. This demo uses the Arduino MEGA board and Wiznet W5100 Ethernet shield (Ethernet module version 2) to perform the actions.

Heat Pump Library – Arduino Mega:

Get the Nabto Arduino Mega2560 library from [https://github.com/nabtodaemon/heatcontrol-arduinomega#heatcontrol-arduinomega] and follow the below installation instructions.

  • Add the library to the Arduino IDE via

Sketch -> Include Library -> Add .ZIP Library

Browse to the folder containing the downloaded library file and add the unabto-arduinomega-sdk-2.1.1.zip (downloaded zip file)

  • Open the HeatPump.ino example via

File -> Examples -> Nabto-Mega2560 -> HeatPump

The sample sketch includes the Nabto class, which encapsulates the Nabto setup. For the sketch to work, the below settings are to be changed. The setting should specify the board’s MAC address (found on the Ethernet board) followed by the unique Device ID and pre-shared key of the device obtained from portal.appmyproduct.com

// Enter device id and pre-shared key from portal.appmyproduct.com
const char* DEVICE_ID = "abc.xyz.appmyproduct.com";
const char* PRE_SHARED_KEY = "4f2a03f29f509035c03bc229ae870849";

(i.e. Sign-up for an AppMyProduct account, Create a product, Create licenses, copy a license Id and License Key into the DEVICE_ID and PRE_SHARED_KEY in the code section mentioned above)

The loop function inside the sketch is used to call the tick() method of the Nabto class that triggers the framework to check for new UDP packets and send responses. The time between ticks should be around 10 milliseconds. This is achieved by a hard delay, but you can also use the time to do application related tasks. For example, the demo uses it to update the brightness of the LED and to simulate the room temperature in the demo application tick function.

Test your device:

After compiling and uploading your HeatPump sketch to the Arduino Mega, it establishes a connection to your Ethernet network and starts the uNabto server. In your serial monitor you should see the following printout:

ArduinoMega-log

Now, using the Heat Control Ionic starter app (also available on google play store) connect your device and verify that the inbuilt LED changes its brightness according to the target heat.

Read more about obtaining the the Heat Control app or how to compile, possible adjust, your own version here https://www.appmyproduct.com/tutorial.html

ArduinoMega-HeatPumpOverview

ArduinoMega-Heatpump

When the Heat Control Ionic app is started for the first time, you need to pair your client app with the device. Then on, the paired device will be automatically discovered if available online.

FAQ:

1. Why is the device not able to communicate with Basestation?

Check the firewall settings on your network.

Ensure your data packets are been transmitted to Basestation and not blocked by your firewall, it must allow outgoing UDP traffic.

2. Device is online, why it is not discovered by my client app?

Ensure the device and the client are connected on the same network.

Search for your device, once discovered pair your client app with the device for the first time use.

Later, the paired device will be listed when the app is started.

3. Device can be accessed locally (on same network), why it cannot be accessed from outside network?

Your device might not be attached to the Basestation. Check the device application logs to locate the below message:

“State change from WAIT_GSP to ATTACHED”

If found, then should be able to connect from outside network. Note, your client must be paired with the device before accessing it.

App My Pi – AppMyProduct Heat Control Demo on Your Raspberry Pi

Have you ever wanted to connect and remote control an embedded device in real-time, like for example your Raspberry Pi? Are you concerned about the security of your communication, but still want to quickly develop high-quality apps? With our IoT application platform AppMyProduct we provide the solution for you!

To give you an impression, we created a Heat Control Demo (app+device source code available on GitHub). This article describes all necessary steps to install and run the demo application on your Raspberry Pi and control it with a smartphone app for iOS/Android – and with a little experience (or some documentation digging) with both HTML5 and Cordova you can extend the demo to whatever RPI remote control purpose you can think of.

rpi-control

Also check out the demo running on a STM32 with FreeRTOS or low-cost ESP8266 WiFi module.

Setup Raspberry Pi

If you already have a Raspberry Pi with working Internet connection, you can skip this step.

Get the Raspbian operating system (lite or with PIXEL) from the official website. Copy the downloaded image to a SD card as described in the documentation.

Use an Ethernet cable to connect your Raspberry Pi to the network, or follow this guide to setup wireless networking.

Install the Heat Control Demo

The device software of our AppMyProduct Heat Control Demo is available on GitHub. The following steps walk you through installing it on your Raspberry Pi.

Build

This step requires git and cmake. I you haven’t installed them already, you should do that now:

sudo apt-get install -y git cmake

To get the source files, clone our GitHub repository and enter the directory:

git clone --recursive https://github.com/nabto/appmyproduct-device-stub.git
cd appmyproduct-device-stub

Create a build folder and enter it:

mkdir build
cd build

Build the demo application:

cmake ..
make -j3

Move the built executable to your home directory:

mv amp_device_stub ~/

Finally, go back to your home directory:

cd

Create Run Script

The demo application takes a couple of parameters, most importantly the unique device id and a licence key. You can obtain both from portal.appmyproduct.com.
You can try to run the demo with the following command (use your parameters):

sudo ./amp_device_stub -d 7pbugghs.smtth.appmyproduct.com -k 985ef2a3de0fe5328cf7c1923b13cbef -N 'RPI' -P 'Raspeberry Pi'

Abort the demo with Ctrl-C.

For a more convenient usage, we create a startup script:

nano run_amp.sh

Copy the following code into the editor (use your parameters):

#!/bin/bash
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
$DIR/amp_device_stub -d 7pbugghs.smtth.appmyproduct.com -k 985ef2a3de0fe5328cf7c1923b13cbef -N 'RPI' -P 'Raspeberry Pi'

Save and close nano with Ctrl-X and confirm with Y and Return.

Make the script executable with:

chmod +x run_amp.sh

Try to run the demo application again using the startup script:

sudo ./run_amp.sh

Run on Startup

If you want the demo to run automatically whenever the Raspberry Pi starts up, open /etc/rc.local with nano:

sudo nano /etc/rc.local

Insert the following line above “exit 0”:

stdbuf -oL /home/pi/run_amp.sh > /tmp/amp.log 2>&1 &

This runs the startup script in background and logs all outputs to /tmp/amp.log.

To test it, reboot you Raspberry Pi using:

sudo reboot

After the reboot, you should be able to see the amp_device_stub process running in the background. Check with:

ps | grep amp_device_stub

You can also see the log file using:

cat /tmp/amp.log

Example log:

11:12:19:331 unabto_common_main.c(127) Device id: '7pbugghs.smtth.appmyproduct.com'
11:12:19:331 unabto_common_main.c(128) Program Release 123.456
11:12:19:331 unabto_app_adapter.c(698) Application event framework using SYNC model
11:12:19:331 unabto_context.c(55) SECURE ATTACH: 1, DATA: 1
11:12:19:331 unabto_context.c(63) NONCE_SIZE: 32, CLEAR_TEXT: 0
11:12:19:331 unabto_common_main.c(206) Nabto was successfully initialized
11:12:19:331 unabto_main.c(85) AppMyProduct demo stub [7pbugghs.smtth.appmyproduct.com] running!
11:12:19:331 unabto_context.c(55) SECURE ATTACH: 1, DATA: 1
11:12:19:331 unabto_context.c(63) NONCE_SIZE: 32, CLEAR_TEXT: 0
11:12:19:331 unabto_attach.c(792) State change from IDLE to WAIT_DNS
11:12:19:331 unabto_attach.c(793) Resolving dns: 7pbugghs.smtth.appmyproduct.com
11:12:19:341 unabto_attach.c(814) State change from WAIT_DNS to WAIT_BS
11:12:19:402 unabto_attach.c(479) State change from WAIT_BS to WAIT_GSP
11:12:19:412 unabto_attach.c(266) ######## U_INVITE with LARGE nonce sent, version: - URL: -
11:12:19:453 unabto_attach.c(580) State change from WAIT_GSP to ATTACHED

Use it!

First, install the AppMyProduct Heat Control Demo on your smartphone (Apple App Store / Google Play / Android APK / Source).

Search for devices in your network. You should see your Raspberry Pi:

rpi-discover

After pairing, you can control the simulated heat pump:

rpi-control

We added a little gimmick for the Raspberry Pi: The device application controls the green on-board LED! Its blink frequency reflects the currently set target temperature. If you deactivate the heat pump, the LED is turned off. (On some Raspberry Pi versions the LED is inverted, hence the LED is permanently turned on in that case.)

Factory Reset

The Heat Control Demo application saves all settings and paired devices in a file called persistence.bin located in the same directory (your home directory in this case). In order to reset these to default, delete the file with

sudo rm persistence.bin

and restart the demo, e.g. by rebooting the Raspberry Pi:

sudo reboot

STM32F746G-DISCO Board with FreeRTOS + AppMyProduct

This is an updated version of a previous post incorporating our new AppMyProduct platform.

When working with STM32 ARM Cortex-M microcontrollers, the free embedded software STM32Cube from ST provides all necessary drivers and a collection of middleware components to reduce the initial development effort. One of the mentioned middleware components is the popular FreeRTOS real-time operating system Nabto is partnering with to create a powerful combined FreeRTOS+Nabto solution.

The launch of our new AppMyProduct application platform helps you quickly develop high-quality apps for remote control of your devices. This article explains the implementation of a demo heat-pump application using Nabto + FreeRTOS on a STM32F746G-DISCO board, which can be controller by our Heat Control Ionic starter app. (A previous blog post described the now deprecated HTML device driver approach.)

nabto+stm32

STM32F74G-DISCO board running the heat-pump demo

What you need

This demo is based and tested on the STM32F746G-DISCO board, but should be portable to similar STM32 boards.

Why Nabto and how does it work?

Have you ever thought about connecting to your STM32 device from outside your local network without nerve wracking router and firewall configurations or slow and intransparent cloud services? Running the uNabto server on your STM32 device you can establish a fast and secure Peer-to-Peer connection using only a static device ID – from everywhere, no matter what is in between.

So how does Nabto do this? The drawing below gives a brief overview. Your STM32 board represents the Device running the uNabto server. As soon as it connects to the internet it identifies itself at the Nabto Basestation, using its unique Device ID. If a Client (e.g. our Heat Control Ionic starter app) wants to connect to the STM32 board, a connect request with the ID is sent to the Basestation, and a direct connection to the device is established.

Screen Shot 2016-02-29 at 13.23.22

Nabto in the STM32Cube Architecture

As mentioned before, the STM32Cube provides a Hardware Abstraction Layer API and a selection of Middleware components. The uNabto server we want to implement builds on top of the STM32Cube middleware, as can be seen in the following diagram:

nabto+stm32cube-architecture.png

The uNabto server itself is divided into two layers:

  • The actual uNabto framework (uNabto SDK)
  • The uNabto Platform Adapter abstraction layer between the uNabto framework and the STM32Cube middleware

Hence, we only need to implement the uNabto Platform Adapter, in order to port the uNabto server and the demo application to the STM32 platform.

Implementing the Platform Adapter

The Platform Adapter acts as link between the generic uNabto framework and the STM32Cube middleware layer, including the LwIP TCP/IP stack which will be used for networking. The LCD Log Utility is used to display the Nabto log output on the screen.

The adapter is divided into single files, as suggested in the Nabto documentation (TEN023 Nabto device SDK guide, chapter 12):

  • unabto_config.h: Basic uNabto configuration
  • unabto_platform_types.h: Defines all necessary uNabto types
  • unabto_platform.h: Platform specific ad-hoc functions
  • network_adapter.c: Init, close, read and write functionality for network data
  • time_adapter.c: Time functions
  • dns_adapter.c: DNS resolving
  • random_adapter.c: Random generator
  • log_adapter.c: Logging

If you are interested in how the platform adapter is implemented in detail, check the adapter files in the Inc and Src directories of the demo on GitHub.

Implementing the Nabto Thread

After creating the Platform Adapter the uNabto Server is ready to use. uNabto is initialized and runs in it’s own FreeRTOS thread. The thread is defined in Src/unabto_main.c:

static void unabto_thread()
{
  const char* device_id = "<DEVICE ID>";
  const char* pre_shared_key = "<KEY>";

  // Init uNabto
  nabto_main_setup* nms = unabto_init_context();
  nms->id = strdup(device_id);

  nms->secureAttach = true;
  nms->secureData = true;
  nms->cryptoSuite = CRYPT_W_AES_CBC_HMAC_SHA256;

  if (!unabto_read_psk_from_hex(pre_shared_key, nms->presharedKey, 16)) {
    NABTO_LOG_ERROR(("Invalid cryptographic key specified", pre_shared_key));
    return;
  }

  if (!unabto_init()) {
    NABTO_LOG_FATAL(("Failed at nabto_main_init"));
  }

  // Init demo application
  demo_init(do_factory_reset);
  demo_application_set_device_name("STM32F746G-DISCO");
  demo_application_set_device_product("ACME 9002 Heatpump");
  demo_application_set_device_icon_("img/chip-small.png");

  // Main loop
  for (;;) {
    unabto_tick();
    osDelay(10);
    demo_application_tick();
  }
}

The code above initializes the server with your unique Device ID and the pre-shared encryption key from portal.appmyproduct.com (insert in highlighted lines). Then the heat-pump demo, which will be briefly described later, is initialized. Finally, an infinite loop repeatedly calls the unabto_tick() method. This triggers the framework to check for new UDP packets and send responses. The time between ticks should be around 10 milliseconds, which is achieved by a hard OS delay and some additional demo specific workload.
To start the Nabto thread we provide a unabto_start() method. It is called by the main program after initializing the TCP/IP stack and either setting a static IP address or obtaining a dynamic IP address from a DHCP server.

void unabto_start()
{
  sys_thread_new("uNabto", unabto_thread, NULL, DEFAULT_THREAD_STACKSIZE, NABTO_THREAD_PRIO);
}

The actual handling of received client requests is implemented in the application_event() callback function. The handler uses the interface definition shared with the client.

application_event_result application_event(application_request* request,
                                           unabto_query_request* query_request,
                                           unabto_query_response* query_response) {

    NABTO_LOG_INFO(("Nabto application_event: %u", request->queryId));
    debug_dump_acl();

    // handle requests as defined in interface definition shared with
    // client - for the default demo, see
    // https://github.com/nabto/ionic-starter-nabto/blob/master/www/nabto/unabto_queries.xml

    application_event_result res;

    switch (request->queryId) {
    case 10000:
        // get_public_device_info.json
        if (!write_string(query_response, device_name_)) return AER_REQ_RSP_TOO_LARGE;
        if (!write_string(query_response, device_product_)) return AER_REQ_RSP_TOO_LARGE;
        if (!write_string(query_response, device_icon_)) return AER_REQ_RSP_TOO_LARGE;
        if (!unabto_query_write_uint8(query_response, fp_acl_is_pair_allowed(request))) return AER_REQ_RSP_TOO_LARGE;
        if (!unabto_query_write_uint8(query_response, fp_acl_is_user_paired(request))) return AER_REQ_RSP_TOO_LARGE;
        if (!unabto_query_write_uint8(query_response, fp_acl_is_user_owner(request))) return AER_REQ_RSP_TOO_LARGE;
        return AER_REQ_RESPONSE_READY;

    case 10010:
        // set_device_info.json
        if (!fp_acl_is_request_allowed(request, REQUIRES_OWNER)) return AER_REQ_NO_ACCESS;
        res = copy_string(query_request, device_name_, sizeof(device_name_));
        if (res != AER_REQ_RESPONSE_READY) return res;
        if (!write_string(query_response, device_name_)) return AER_REQ_RSP_TOO_LARGE;
        return AER_REQ_RESPONSE_READY;

    case 11000:
        // get_users.json
        return fp_acl_ae_users_get(request, query_request, query_response); // implied admin priv check

    case 11010:
        // pair_with_device.json
        if (!fp_acl_is_pair_allowed(request)) return AER_REQ_NO_ACCESS;
        res = fp_acl_ae_pair_with_device(request, query_request, query_response);
        debug_dump_acl();
        return res;

        // [...]
    }
}

For more details on the Heat-Pump demo application please review the source in Src/unabto_application.c.

Hands-On!

Enough theory. Let’s try out the demo! It is available as System Workbench for STM32 (SW4STM32) project here on GitHub. Simply follow the instructions in the README.

The board should print the Nabto log on the LCD screen, as shown on the picture in the beginning of the article.

Now, connect to your device using the Heat Control Ionic starter app and see the parameters on the bottom of the LCD display change according to your app inputs. The simulated room temperature slowly converges to the target temperature. You can update the current room temperature in your app by pressing Refresh.

amp_stm32_app

The device settings are persistently stored in the STM32 flash memory. If you want to perform a “factory reset”, press the User button on the board during startup/reset. You should see FACTORY RESET printed to the display.

Java Agent for Cassandra Metrics Export to Prometheus

At Nabto we are using many technologies in our platform, one of them is Cassandra. We are monitoring Cassandra with the following setup setup: Cassandra + Prometheus JMX Exporter -> Prometheus -> Grafana. This setup has a component which does not meet our requirements for a production setup: The Prometheus JMX Exporter uses too much memory and CPU for the simple task it has to do.

To mitigate this problem we have created a new Java Agent which directly exports the Dropwizard metrics from the Cassandra core through the Prometheus Dropwizard Exporter and serves the metrics through a resource limited Jetty HTTP server.

The code and documentation can be found on GitHub https://github.com/nabto/cassandra-prometheus

 

ESP8266 WiFi Module + AppMyProduct

This is an updated version of a previous post incorporating our new AppMyProduct platform.

The ESP8266 is a low-cost WiFi module that can be programmed directly like a microcontroller. Already thinking of your next Internet of Things project?

While an available Arduino library allows a quick start, there is still one problem to overcome: How to access your ESP8266 from outside your home network without nerve-racking router and firewall configurations or heavy cloud services? Running the uNabto server on your ESP8266, you can establish a fast and secure Peer-to-Peer connection using only a static ID – from everywhere, no matter what is in between.

What hardware you’ll need

IMG_20160303_153803

Adafruit HUZZAH ESP8266 Breakout

IMG_20160303_153830

USB to TTL Serial Cable

This project is tested on an Adafruit ESP8266 board. It’s not the cheapest you can get, but very prototyping friendly. You can put it on a breadboard and it has a 3.3V voltage regulator onboard. Of course, you can also use a different ESP8266 module. Wikipedia provides an extensive list of available modules.

Regarding the USB to TTL Serial Cable (get it e.g. here), there are no special requirements when using the Adafruit module. If you use a different module, make sure the voltage levels match, since most boards only accept 3.3V.

Solder the pin header to your ESP8266 board and connect it to your USB to TTL adapter using 4 wires:

wiring

What software you’ll need

This project is tested on Ubuntu 16.04 but should work on any OS supported by the Arduino IDE.

We want to program the ESP8266 directly, instead of using any higher level interpreters available for the module. Adafruit provides a nice tutorial on how to setup your Arduino IDE accordingly.

If everything is running, you should be able to compile the following simple sketch and upload it to your ESP8266. This will make the onboard LED blink every second.

const int led = BUILTIN_LED;

void setup() {
  pinMode(led, OUTPUT);
}

void loop() {
  digitalWrite(led, HIGH);
  delay(500);
  digitalWrite(led, LOW);
  delay(500);
}

How the Nabto platform works

So far, so good. But how does the Nabto platform actually work? The drawing below gives a brief overview. Your ESP8266 module represents the Device running the uNabto server. As soon as it connects to the internet it identifies itself at the Nabto Basestation, using its unique ID. If a Client wants to connect to the ESP8266, a connect request with the ID is sent to the Basestation, and a direct connection to the device is established. A client can be a native app or use an abstraction framework like our Heat Control Ionic starter app used in this demo.

nabto-platform-basics

Get more information on the AppMyProduct platform and the Client/Device SDKs on portal.appmyproduct.com.

The uNabto Platform Adapter

The uNabto server is divided into two layers:

  • The actual uNabto framework (uNabto SDK)
  • The uNabto Platform Adapter abstraction layer between the framework and the Native Platform

unabto-platform-adapter

Hence, we only need to implement the uNabto Platform Adapter, in order to port the uNabto server to the ESP8266 module.

Implementing the Platform Adapter

The Platform Adapter acts as link between the generic uNabto framework and the Arduino platform, including the ESP8266 WiFi library. The adapter is divided into single files, as suggested in the Nabto documentation (TEN023 Nabto device SDK guide, section 12):

  • unabto_config.h: Basic uNabto configuration
  • unabto_platform_types.h: Define all necessary uNabto types
  • unabto_platform.h: Platform specific ad-hoc functions
  • network_adapter.cpp: Init, close, read and write functionality for network data
  • time_adapter.cpp: Time functions
  • dns_adapter.cpp: DNS resolving
  • random_adapter.cpp: Random generator
  • log.cpp: Logging

If you are interested in how the platform adapter is implemented in detail, check the adapter files in the src directory of the library on GitHub.

Using the Library

Get the Nabto ESP8266 Arduino library from https://github.com/nabto/unabto-esp8266-sdk and follow the installation instructions.

An example sketch can be found in

File -> Examples -> Nabto-ESP8266 -> HeatPump

The sample sketch includes the Nabto class, which encapsulates the Nabto setup. First, some settings are made. This includes the WiFi SSID and password, followed by the unique Device ID and preshared key of the device obtained from portal.appmyproduct.com

// Enter ssid and password of your WiFi network
const char* WIFI_SSID = "<SSID>";
const char* WIFI_PASSWORD = "<PASSWORD>";

// Enter device id and pre-shared key from portal.appmyproduct.com
const char* DEVICE_ID = "<DEVICE ID>";
const char* PRE_SHARED_KEY = "<PRE-SHARED KEY>";

The setup function is used to init the LED pin, the Serial module, and connect to the WiFi. In line 91 you can see how the Nabto module is initialised. After the Nabto version is printed our heatpump demo is initialized.

void setup() {
    // Initialize Serial
    Serial.begin(115200);

    // Wait 2s for button press to do factory reset
    pinMode(0, INPUT_PULLUP);
    bool factory_reset = false;
    while(millis() < 2000) {
      if(digitalRead(0) == LOW) {
        Serial.println("FACTORY RESET");
        factory_reset = true;
        break;
      }
    }

    // Initialize built-in led
    pinMode(LED_PIN, OUTPUT);
    analogWrite(LED_PIN, PWMRANGE);

    // Initialize WiFi
    WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
    Serial.print("Connecting to WiFi..");
    while (WiFi.status() != WL_CONNECTED) {
        Serial.print(".");
        delay(500);
    }
    Serial.println("done");

    // Initialize Nabto
    Serial.println("Init Nabto...");
    Nabto.begin(DEVICE_ID, PRE_SHARED_KEY);

    // Print Nabto version
    char versionString[10];
    Nabto.version(versionString);
    Serial.print("Nabto v");
    Serial.print(versionString);
    Serial.println(" running");

    // Initialize demo application
    demo_init(factory_reset);
    demo_application_set_device_name("ESP8266");
    demo_application_set_device_product("ACME 9002 Heatpump");
    demo_application_set_device_icon_("img/chip-small.png");
}

The only thing that needs to be done in the loop function is to call the tick() method of the Nabto class. This triggers the framework to check for new UDP packets and send responses. The time between ticks should be around 10 milliseconds. This is achieved by a hard delay, but you can also use the time to do application related stuff. For example, we use it to update the brightness of the LED and to simulate the room temperature in the demo application tick function.

void loop() {
    Nabto.tick();
    demo_application_tick();
    delay(10);
}

The actual handling of received Nabto messages from the client is implemented in the application_event(..) function. The handler uses the interface definition shared with the client.

pplication_event_result application_event(application_request* request,
                                           unabto_query_request* query_request,
                                           unabto_query_response* query_response) {

    //NABTO_LOG_INFO(("Nabto application_event: %u", request->queryId));
    //debug_dump_acl();

    // handle requests as defined in interface definition shared with
    // client - for the default demo, see
    // https://github.com/nabto/ionic-starter-nabto/blob/master/www/nabto/unabto_queries.xml

    application_event_result res;

    switch (request->queryId) {
    case 10000:
        // get_public_device_info.json
        if (!Nabto.write_string(query_response, device_name_)) return AER_REQ_RSP_TOO_LARGE;
        if (!Nabto.write_string(query_response, device_product_)) return AER_REQ_RSP_TOO_LARGE;
        if (!Nabto.write_string(query_response, device_icon_)) return AER_REQ_RSP_TOO_LARGE;
        if (!unabto_query_write_uint8(query_response, fp_acl_is_pair_allowed(request))) return AER_REQ_RSP_TOO_LARGE;
        if (!unabto_query_write_uint8(query_response, fp_acl_is_user_paired(request))) return AER_REQ_RSP_TOO_LARGE;
        if (!unabto_query_write_uint8(query_response, fp_acl_is_user_owner(request))) return AER_REQ_RSP_TOO_LARGE;
        return AER_REQ_RESPONSE_READY;

    case 10010:
        // set_device_info.json
        if (!fp_acl_is_request_allowed(request, REQUIRES_OWNER)) return AER_REQ_NO_ACCESS;
        res = Nabto.copy_string(query_request, device_name_, sizeof(device_name_));
        if (res != AER_REQ_RESPONSE_READY) return res;
        if (!Nabto.write_string(query_response, device_name_)) return AER_REQ_RSP_TOO_LARGE;
        return AER_REQ_RESPONSE_READY;

    case 11000:
        // get_users.json
        return fp_acl_ae_users_get(request, query_request, query_response); // implied admin priv check

        // [...]
    }
}

Test your device

After compiling and uploading your HeatPump sketch to the ESP8266, it establishes a connection to your WiFi network and starts the uNabto server. In your serial monitor you should see the following printout:

Connecting to WiFi........done
Init Nabto...
Device id: 'n7j4qebq.hyr7o.appmyproduct.com'
Program Release 123.456
Application event framework using SYNC model
SECURE ATTACH: 1, DATA: 1
NONCE_SIZE: 32, CLEAR_TEXT: 0
Nabto was successfully initialized
Nabto v123.456 running
SECURE ATTACH: 1, DATA: 1
NONCE_SIZE: 32, CLEAR_TEXT: 0
State change from IDLE to WAIT_DNS
Resolving dns: esp8266test1.demo.nab.to
State change from WAIT_DNS to WAIT_BS
State change from WAIT_BS to WAIT_GSP
########    U_INVITE with LARGE nonce sent, version: - URL: -
State change from WAIT_GSP to ATTACHED

Now, connect to your device using the Heat Control Ionic starter app and see the LED change its brightness according to the target heat.

amp_esp8266_app

Jorjin Ameba Arduino Library

Today we will have a look at the Arduino-like Jorjin Ameba board with built in support for WiFi and NFC. Taking the pricetag of 25$ into account makes this an easy choice for your IoT projects!

For this post I created a Jorjin Ameba library which can be imported directly into the Arduino IDE seamlessly, thus making the classic blink example a breeze.

We will walk through the following steps

  • Specs of jorjin board and “special features”
  • How to run the example
  • Demo

The Jorjin board

board

Overview of the Jorjin Ameba WiFi board.

The Jorjin Ameba WiFi board has built-in WiFi and  NFC thus making it very much prepared for IoT.

As can be seen from the above image, the Ameba board has an Arduino-like form factor with a similar pinout. For the simple LED blink example, we will be using a digital pin.

Getting the example up and running

To run the example we need to install the Ameba board libraries and the Ameba specific uNabto files. We also need a device name and key, both of which can be created at developer.nabto.com

Furthermore an LED (and appropriate resistor) is needed.

Step 1: Download General Ameba Libraries install

Follow the steps laid out here to install the general Ameba libraries into the Arduino IDE OR read it right here:

Open up the Arduino IDE. Click File -> Preferences and then copy this

https://github.com/Ameba8195/Arduino/raw/master/release/package_realtek.com_ameba_index.json

into the Additional Boards Manager URLs input field. Click OK

Now click Tools –> Board: -> Boards Manager

Here we search for Ameba and click Install

The Jorjin Ameba board Arduino libraries are now installed

Step 2: uNabto libraries install

There are two ways of getting the uNabto libraries.

Using git

git clone --recursive https://github.com/nabto/unabto-jorjin-sdk

then change directory to the unabto-jorjin-sdk folder and run

bash Make.sh

OR download the release zip file unabto-jorjin-sdk.zip

No matter which way, we can now add the library to the Arduino IDE via Sketch -> Include Library -> Add .ZIP Library... and then browse to and add the unabto-jorjin-sdk folder that was just downloaded.

Run the example

We can now open the LightSwitch.ino example by going to File -> Examples -> Nabto-Ameba -> LightSwitch

Here we need to enter the SSID and password of the wireless network we want to attach to. Furthermore we need to input the device name and key we created at developer.nabto.com

Finally we need to remember to wire the LED and resistor up like so (try switching pins around if it does not work at first)

jorjin_wiring

Wiring of the LED. Image taken from the Ameba documentaion, available from here (signing up is required)

When the example is compiled and uploaded to the device we simply need to press the reset button (see the above image). The board should now reset, connect to the specified wifi network and run uNabto.

We can now control the LED by going to devicename.demo.nab.to in the Nabto client of your choice. (devicename is the unique name created at portal.nabto.com). Simply use the guest account for this demo. We can now control the LED by moving the slider.

Demo

As always, the full code can be found at Github. If you feel like experimenting with the many uses of the Nabto framework, please visit our developer portal where you can manage up to 10 devices for free !

The Roomba (Part 2 of 2)

Please read Part 1 to read about the preliminary steps in the Roomba saga.

This part will be the fun part where we turn the Roomba into a fully fledged remote controlled vehicle fitted with realtime video feed as well!

If at any time you feel like trying this yourself, you can sign up for a free developer account or/and read more at developer.nabto.com.

In details, this part will be dealing with

  • Turn the Roomba into a remote controlled car
  • Add soundeffects
  • Add sensor data
  • Add video streaming

The RC Roomba

Turning the Roomba into a remote controlled car is really easy once the groundwork is done correctly (see Part 1).

All we need are the new opcodes given to us in the documentation:

  • Drive straight (20 cm/s)
    • Using W
    • {137, 0, 255, 128, 0}
  • Reverse (20 cm/s)
    • Using S
    • {137, 255, 0, 128, 0}
  • Rotate counterclockwise
    • Using A
    • {137, 0, 255, 0, 1}
  • Rotate clockwise
    • Using D
    • {137, 0, 255, 255, 255}

This is all we need on the device side of things.
For the html part I simply chose to create a text input field which detects when a certain character is pressed in accordance with the keys given above. Say, for example, that you press W. The Roomba will now drive in a straight line until the user sends a new command.
If a key is pressed which does not have a predefined action, then the Roomba will stop driving. I usually use space to stop.

Adding sound effects

It is possible to store melodies in the Roomba. On our specific model, 581, we can store 4 melodies with 16 notes each (that storage!). From the documentation we find the serial sequence to be

[140] [Song Number] [Song Length] [Note Number 1] [Note Duration 1] [Note Number 2] [Note Duration 2].etc.

I decided to add a reverse sound, the DSB sound and a Dixie Horn style, River Kwai March (warning: loud) sound. These were added with the following codes

  • Reverse sound
    • {140, 1, 6, 84, 32, 30, 32, 84, 32, 30, 32, 84, 32, 30, 32}
  • DSB sound
    • {140, 0, 3, 74, 40, 75, 20, 70, 32}
  • Dixie Horn sound
    • {140, 3, 12, 86,10, 83,10, 79,10, 79,10, 79,10, 81,10, 83,10, 84,10, 86,10, 86,10, 86,10, 83,10}

The songs can then be played back by this sequence [141][Song Number], like this

  • Reverse sound
    • Played when the Roomba goes in reverse
    • {141, 3}
  •  DSB
    • Using K
    • {141, 0}
  • Dixie Horn
    • Using L
    • {141, 3}

Sensor data

The Roomba has 58 different sensor data packets, which all return hex values. For this I want to query

  • Battery temperature
    • Packet ID 24
    • Data bytes: 1, signed.
    • Returns the temperature of the Roomba’s battery in degrees Celsius.
  • Battery charge
    • Packet ID 25
    • Data bytes: 2, unsigned
    • Returns the charge level in mAh
  • Battery capacity
    • Packet ID 26
    • Data bytes: 2, unsigned
    • Returns battery capacity in mAh
  • Charging state
    • Packet ID 34
    • Data bytes: 1, unsigned
    • Returns 1 if charging, 0 otherwise

We can query all these with a single sequence

[149][Number of Packets][Packet ID 1][Packet ID 2]...[Packet ID N]

which look likes this in our case

{149, 4, 24, 25, 26, 34}

Referencing the above Data bytes gives us a total sensor byte size of 6 bytes. The battery temp and charging state can be taken directly, since they only have 1 data byte. Charge and capacity need some attention since we need to bitwise left shift on the high (first) byte before we can get a decimal number. In C this is done like so

Number = [1]<<8 | [2]

We then calculate the battery level by simply doing

bat_level = 100 * (charge/capacity)

I then created a button for our html code which output charging state, battery level and battery temperature, when pressed.

Video streaming through Nabto tunnel

For this we selected to use a Raspberry Pi Camera Module since we already had one at the office. I then read about various ways of streaming the output and display it on a webserver. I tried many different approaches but eventually settled on RPi Cam Web Interface since it gave the best compromise between low latency, low bandwith usage and “high” resolution. Furthermore, these settings are fully costumisable and the user can thus change values for their needs. An extra feature is the ability to run motion detection and to take 2592 x 1944 pixel static images or 1080p @ 30fps, 720p @ 60fps  and 640x480p 60/90 video recording.

Setting up a webserver on our Pi

To get the video webserver up and running on our Pi is fairly easy when following the instructions laid out here.

sudo apt-get update
sudo apt-get dist-upgrade
git clone https://github.com/silvanmelchior/RPi_Cam_Web_Interface.git
cd RPi_Cam_Web_Interface
chmod u+x *.sh
./install.sh

The installation prompt will have some adjustable settings such as whether to run an apache or nginx webserver. Choose whatever you prefer. I chose to change the default port from 80 to 90 to avoid conflict with any other service running on our Pi. After installation a webserver should now be running. We can access it on our local network by going to pi_ip:port_chosen/html

If no video is running, try issuing ./start.sh or do an update by ./update.sh.

The default settings worked very well for me but if you happen to have a very low bandwith internet connection (or running this on one of the earlier Pi versions) you can try to adjust the bandwith settings.

Tunneling

To make our video feed available from anywhere, we will be running a Nabto tunnel on our Pi as well. Compiling a Nabto tunnel is very easy, simple issue these commands one line at a time on the Pi

cd unabto/apps/tunnel
cmake .
make

which we can execute by doing

./unabto_tunnel -d "id".demo.nab.to -s -k "key"

where the id and key are generated at developer.nabto.com

Finally we need to run the other end of the tunnel on our client machine. I will present how to do this using our simpleclient_app solution which works on Linux, Windows and Mac OSX. Furthermore I will walk through how to use the special Tunnel manager tool for Windows. All this can be downloaded from developer.nabto.com where we also need the common Nabto Client SDK Resources.

Command line

On our client machine, combine the Simple Client and Client SDK Resources such that the structure look like this
./bin/simpleclient_app and ./share/nabto/users/guest.crt and then issue

./simpleclient_app -q "id".demo.nab.to --tunnel client_port:localhost:device_port

which should output a few lines ending with either
tunnelStatus LOCAL
or
tunnelStatus REMOTE_P2P
depending on if the connection to the Pi is local or remote.

Tunnel Manager for Windows

Grab a copy of Tunnel Manager and input

"id".demo.nab.to into "Server"
127.0.0.1:client_port into "Local endpoint"
127.0.0.1:device_port into "Remote endpoint"

Which can look something like this

tunnelmanager_windows

Tunnel Manager for Windows. Change the Server name, Local endpoint and Remote endpoint to whatever you chose earlier on.

The tunnel is now up and running on the Pi and we are connected to from the client.

Let’s have some fun!

Everything is now setup. On our device side we are running a Roomba remote ./unabto_raspberry and a ./unabto_tunnel.

On the client side we are either running no tunnel (if we do not need video streaming) or either a command line ./simpleclient_app or the Tunnel Manager for Windows.

We can now finally access our uNabto Roomba webpage as we did in Part 1 where we will be greeted with this.

roomba_video_html

uNabto Roomba remote control html device driver

As stated below the image, clicking it will open a new window showing us the tunneled stream from the webserver running on the Pi.
Just below the image is a button for reading sensor data and below that I placed the input field for controlling the uNabto Roomba.

All that is left is to test it out!

If you feel like doing the same thing, please feel free to check out the code on Github and sign up for a developer account at developer.nabto.com, it is all free and you can create and manage 10 devices. This is also where you can find the SDKs and other Nabto software!