Connecting a camera to the Pi

Testing the camera on the Raspberry Pi

Advertisements

Ok, so today I got my new nifty gadget.  A camera for the Pi!  So here’s the unboxing.

I got it in a neat paper bag like this one :

16832778_2225601784330794_64021547_o.jpg

This is what it looks like.

16880823_2225601770997462_1406325668_o.jpg

I opened the casing and connected it. It’s fairly easy.  Just pull up the handles, put it in, and push the handles back down.

16879954_2225601744330798_1761360833_o.jpg

Now to configure this.  In order to do this, just log on to the pi and start raspi config :

sudo raspi-config

locate the Interfacing options

Schermafbeelding 2017-02-20 om 19.30.54.png

Locate the camera option

Schermafbeelding 2017-02-20 om 19.31.12.png

Enable it

Schermafbeelding 2017-02-20 om 19.31.22.png

and reboot.

Schermafbeelding 2017-02-20 om 19.31.39.png

After the reboot sequence has finished, it’s time to test this.  There are actually 2 command line tools you can use to capture a still image and a video image.  The first one is called raspistill.  It’s got tons of options really.  You should really take a look at them.  But the easiest way to do a test is using this command :

raspistill -o me.jpg

Where the -o parameter sets the output filename.

Schermafbeelding 2017-02-20 om 19.37.04.png

So here it is :

Schermafbeelding 2017-02-20 om 19.38.53.png

The second one is raspivid

Here’s the syntax :

raspivid -o test.h264 -t 3000

in this command the .h264 file is the output. the -t parameter sets the number of seconds to record. It’s expressed in milliseconds, so 3000 means 3 seconds.

Schermafbeelding 2017-02-20 om 19.43.19.png

Good luck testing this!

Starting of with mirror development

Choosing a suitable development environment as well as deployment options for the magic mirror development.

We have come at a point in our mirror development where we can start looking at integration of the Alexa Voice Services with the magic mirror software, and adding extra functionality to both AVS and the smart mirror.

But for that we need an IDE, or at least an editor for Javascript / CSS / HTML.

I’ve pondered on several options.  There quite some good editors out there.  Some of which are free like atom, brackets and some of which are paying like Sublime Text and Webstorm.

I didn’t feel like paying for an editor, so that leaves the free alternatives.  Looking into those alternatives, however, none really work on a Raspberry Pi.  Or not out of the box, or with a few tweaks and workarounds.   According to me, not a future proof option.

As an alternative there is an option for full fledged IDE’s, two of which are Eclipse and NetBeans.  We’ve already installed NetBeans on the Pi for the Java development, but we’ve installed the SE version, which is the lowest version, with the smallest footprint.

Installing a higher version of NetBeans could be an option, but when editing and debugging the Java Client I found that the speed of the Raspberry Pi just doesn’t cut it when it comes to a fluent way of working with the IDE.  I know from experience (I used to make android apps as well) that eclipse is a very “heavy” IDE with lots of configuring and a large footprint.  Seeing what NetBeans does, I’m not even considering it.

I considered VIM.  Yes I did.  I seriously did.  There’s a lot of controversy about the tool.  Some couldn’t live without, and some find it sh*t.
I have to say I can understand the pro’s and cons.  I’ve seen some good tutorials on the tool, and I also read the documentation (vimtutor).  The shear power of the commands, the macro’s, the plugins etc make it a truly powerful editor.  This is at the same time its biggest problem.  There are so many options, and to be frank, there is quite a steep learning curve in getting to know all of the commands before you can gain an advantage is development time by using all the shortcuts.

I figured that it’s perfect for people who are developing all of the time, but for someone like me, that will do it only occasionally, a more graphical and intuitive interface would have more benefits.

Another disadvantage is the lack of debugging functionality. You really need to know your syntax and possibilities pretty well.  There’s highlighting and code completion (if you install the plugins), but that doesn’t prevent you from writing “dumb” code.

These two things combined made me search for an other solution.

 

I figured that it was an advantage for me to be running all this from a Mac.  In the end, I can run the mirror software on my mac as well.  I’m not able to run the java code of the AVS client (yet), but I will worry about that later.

The big advantage to be editing and running the mirror software on the Mac would be speed and debug ability.  Since we already installed atom before, I will keep on using this editor.  It’s fast and has quite some cool plugins to make my life easier.

The disadvantage here, however, is deployment to the Pi.  How to get the latest software to the Pi after developing on the Mac.

I have been thinking about that, and there are several command line tools that could help

  • rsync
  • sftp
  • scp

All of these have pro’s and cons as well, but what they have in common is the setup cost.  I would need to create scripts, setup authentication, etc.  That’s quite some effort while I’m not planning on deploying just yet.  And if I do, it won’t happen that often.  That’s why I’ve abandoned this idea, and I will by copying over the changed files from the finder, whenever I want to deploy.  There aren’t that many html, css and js files, so it’s no big hassle to do it manually.

So there we have it: a consensus.  Time to get cracking!

 

Creating desktop icons

Create desktop icons to start the smart mirror software.

In this article I’ll be describing a way to quickly start the applications with a shortcut on the desktop.  For a good working smart mirror I will need to start at least the following applications :

  • The smart mirror software itself
  • The Java Client
  • The Wake word agent

In order to avoid hooking up a keyboard every time I want to start them, I will create desktop icons.  That way I’ll only have to connect a mouse to start the applications.

Turns out that’s quite easy.

Log on to the Raspberry Pi with ssh, or take over the screen using VNC, and open a terminal on the desktop.

Next, navigate to the Desktop directory :

cd Desktop

Create a new file called client.desktop :

nano client.desktop

This will automatically open the nano editor.  There is a fixed format in which these files are structure.  First of all, it needs to start with [Desktop Entry], and the settings are key value pairs separated with an equals sign.  The settings to start the java client are as follows :

[Desktop Entry]
Encoding=UTF-8
Version=1.0
Type=Application
Path=/home/pi/smartmirrorproject/alexa-avs-sample-app/samples/javaclient
Exec=mvn exec:exec
Terminal=false
Name=client

A few settings are important :

  • Always use “Application” as the Type.
  • Set a path.  It is the working directory for the application where it can find its configuration files etc.
  • Set Terminal=false to avoid starting a terminal every time.

More info on the other settings can be found here.

I’ve pasted the settings for the smart mirror and the wake word engine below.

[Desktop Entry]
Encoding=UTF-8
Version=1.0
Type=Application
Path=/home/pi/smartmirrorproject/Smartmirrorproject/
Exec=npm start
Terminal=false
Name=mirror
[Desktop Entry]
Encoding=UTF-8
Version=1.0
Type=Application
Path=/home/pi/smartmirrorproject/alexa-avs-sample-app/samples/wakeWordAgent/src
Exec=lxterminal -e "/home/pi/smartmirrorproject/startup.sh"
Terminal=false
Name=wwa

Notice above how the “Exec” value is configured.  For the wake word engine a script is started instead of an executable.  In order to show its progress in a terminal I chose to start is as a command with inside a terminal by using the lxterminal command.

The result looks like this :

Schermafbeelding 2017-01-14 om 20.11.33.png

 

Raspberry Pi sound output

How a Raspberry Pi is configured to switch between a 3.5mm Jack sound output to HDMI sound output.

So, up until now, I was using a headphone to listen to Alexa’s responses to my questions.  It was time to experiment with sound output. Check out my Amazon Voice Services setup procedure here.

I figured, the easiest way to get sound is through the HDMI interface to the screen.  The screen I’m using contains speakers, so no extra external speakers would be necessary.

There are 2 things you need to do to force the Raspberry Pi to relay the sound output to the HDMI interface.

Log on to the Pi using ssh and type

sudo raspi-config.

Navigate to the advanced options, and choose “Audio”.

Schermafbeelding 2017-01-12 om 18.38.35.png

Select the third option : “Force HDMI”.

Schermafbeelding 2017-01-12 om 18.39.01.png

Confirm, and get out of the interface.

Next, go to the root of the device by typing

cd /

navigate to the boot directory

cd boot

Edit the “config.txt” file.

sudo nano config.txt

Search for the “hdmi_drive=2” setting.  It will probably be commented out using a hash tag in front of it.  Just remove the hash tag.

Schermafbeelding 2017-01-12 om 18.51.49.png

Ctrl-x y enter to get out of the interface.

Reboot the Pi

sudo reboot

Time to do some testing.  If you also have a monitor, go ahead and test Alexa. In my case there was a problem.  The sound worked all right, but there was a lot of latency.  So much latency, the beeping sound Alexa makes when it indicates it has started listening couldn’t be heard.  The first two words (on average) of Alexa’s response were missing as well.

I decided to reverse the settings above (insert the hash tag, force the 3.5mm Jack) and I bought a 14 euro speaker set in the local grocery store.  It’s powered using USB and connects to the 3.5 mm Jack.  It appears to work perfectly.  The problem will probably be integrating them in the mirror.  They are the smalles speakers I sound find, but still wide enough to have a hard time integrating them.  But that’s a bridge I’ll be crossing later.

Enjoy!

Setting up Alexa – Part 2

The second part on a series of setting up Alexa Voice Services on a Raspberry Pi.

In this second part of the Alexa setup I will be installing the software onto my Pi.

Connect to the Pi using ssh.

Move into the “smartmirrorproject” directory

cd smartmirrorproject

Amazon kindly delivers a Java sample application you can pull from GitHub that lets you interact with Alexa.  You don’t have to write all of the code yourself, but you can edit it.  And that just what we’ll do later on.

But first things first.  I say that a lot, don’t I 🙂

Pull the Alexa software onto the Pi by typing

git clone https://github.com/alexa/alexa-avs-sample-app.git

Amazon delivers an installation script that does all of the hard work (installation work that is) for you.  However, it needs some information.  It needs the keys that were generated for you in the first step of the Alexa configuration.

Change directory into the alexa-avs-sample-app.

cd alexa-avs-sample-app

Open up the installation script by typing

nano automated_install.sh

At the beginning of the file you can copy-paste your keys into the placeholders.

The ProductID in our case is “SmartMirror”.  The ClientID and ClientSecret can be found in the security profile on your Amazon Developer Portal.

Ctrl-x Y enter to get out of nano.

Time to start the installation script by typing

. automated_install.sh

It’s an executable file, so it will start automatically.  Next, you are prompted with a number of questions.  I answered “y” to all of them.  One of them is the question if I was going to attach speakers to the 3.5mm jack output of the Pi.  I chose yes, but later on we can change that to re-route the output through the HDMI output should the screen contain speakers.

Installing this will take a while.

For the next steps it’s important you use VNC to see the desktop, and don’t use ssh.

Open a terminal and navigate to

cd smartmirrorproject/alexa-avs-sample-app/samples/companionService

type

npm start

to start the application.  You will see it is listening on port 3000.

The companion service is responsible for the authentication in this case.  Next, we want to start the Java application that will take care of sending to audio to AVS and receiving it back.

Open a second terminal and type

cd smartmirrorproject/alexa-avs-sample-app/samples/javaclient

then type

mvn exec:exec

This will start a tool called “maven” which will execute a series of commands to start the Java Client application.

When the application starts it will prompt you with a dialog box.

Schermafbeelding 2017-01-07 om 22.23.22.png

Click Yes to open in your default browser.   Pay attention : a new dialog will appear asking you to click OK.  Don’t do that yet.  I repeat.  Don’t do that yet.

In the browser window you will need to provide your developer account credentials.  It will probably ask you to do it twice (using a captcha) and click OK.  After doing so, the window will say “Device Tokens are ready”.

At that time, click OK in the window you couldn’t press OK on a minute ago.

A code will automatically be entered in the java application.

Schermafbeelding 2017-01-07 om 22.25.23.png

Click “listen” to activate Alexa.  Should work. 🙂

Lastly we can activate the wake word engine.  The wake word engine is yet another application you need to start, that constantly listens for the word “alexa”, and if it heres it, it will automatically click the “listen” button for you.  Isn’t that the coolest thing.

Open up a third terminal window, and go to the directory

cd smartimirrorproject/alexa-avs-sample-app/samples/wakeWordAgent/src

The wake word engine comes with 2 different technologies you can use : sensory and kitt_ai.

The first one will stop working after 90 days, so we’ll go with kitty_ai.

Start the process by typing :

./wakeWordAgent -e kitt_ai

Mind the “./” at the beginning.  It’s important.

Once started, you can all Alexa and ask anything you like.

Now here’s the sad part.  These last parts are steps you need to take every single time you restart the Pi.  It requires authentication every time.

Amazon states in its documentation the companion service that is started in the above steps is just a demo service, but it’s possible to create your own application that handles the authentication without human interaction.  That’s what we’ll be trying to do.  That way, Alexa can be started every time the Pi boots.  Things to think about!

 

 

Setting up Alexa – Part 1

The first part on a series of setting up Alexa Voice Services on a Raspberry Pi.

Allright, now for the tricky part.

Time to try and set up Alexa on my Pi.

First things first.  Navigate to https://developer.amazon.com

Log in with your amazon developer account.  If you don’t have one, create one.  It’s free.

Navigate to the “Alexa” tab, and choose Alexa Voice Services.

Schermafbeelding 2017-01-07 om 20.30.32.png

Register a product type, and choose “Device”.  After all, we’re integrating Alexa into our Raspberry Pi, which is a device.

Schermafbeelding 2017-01-07 om 20.32.24.png

We are taken into a wizard.    Think of a device type ID.  That’s the unique identifier of all the devices you’re going to create.  I called it “SmartMirror”.  It also needs a name.  I chose “Smart Mirror” with a space between the words.

Schermafbeelding 2017-01-07 om 20.33.36.png

In the next tab you are given a Security Profile ID, a Client ID and a Client Secret.  Those strings are unique, and you will use them to connect to the service online.

Schermafbeelding 2017-01-07 om 20.34.58.png

Go to the “Web Settings” tab, and click “Edit” at the right hand side.

Schermafbeelding 2017-01-07 om 20.37.23.png

Click “Add another” at the “Allowed Origins”.

https://localhost:3000

Do the same with the “Allowed return URLs” with the following :

https://localhost:3000/authresponse

Save.   The end result should look like this.

Schermafbeelding 2017-01-07 om 20.40.04.png

Next, you will be asked to choose a some extra settings like a category, description and commercialisation details.  Just fill in what you want.

Schermafbeelding 2017-01-07 om 20.41.07.png

I don’t have Amazon music, so when the question pops if I want to use it, I chose “No”.

Schermafbeelding 2017-01-07 om 20.42.08.png

That’s it for part one.  We’ve now successfully registered out Raspberry Pi project with Amazon.  It can start accepting calls from our  Pi.

Changing the physical screen resolution

How to change the physical screen resolution on the Raspberry Pi.

Next, I want the same effect on the physical screen.  First changing the resolution.  That’s easy.

Log on to the Pi using ssh.

type the following command :

sudo raspi-config

Locate number 7, “Advanced Options”

Schermafbeelding 2017-01-07 om 19.53.57.png

Choose “Memory Split”

Schermafbeelding 2017-01-07 om 19.54.18.png

Enter 128 as the new memory capacity of the GPU.  This way, a larger portion of the on-board memory of the Pi is reserved from showing things on the screen.  It’s important for larger resolutions.

Schermafbeelding 2017-01-07 om 19.54.31.png

Next, to change the resolution, choose “Resolution” and choose your preferred resolution.

Close the raspi-config utility and reboot by typing

sudo reboot

After the Pi reboots, it’s time to change the resolution.

Connect to the Pi using ssh.

Go to the root of the drive by typing

cd /

change to the boot directory by typing

cd boot

edit the “config.txt” file by typing

sudo nano config.txt

go to the bottom of the screen and type

display_rotate=1

to rotate the screen by 90°.

Ctrl-x and Y to save the file.

Reboot by typing

sudo reboot

Connect a physical screen and see the result.