Sheldon - A Telepresence Robot

Introduction

The Problem

Every year during the Christmas holidays, we get together with friends to celebrate the coming New Year. While most can make it, some live in distant cities and are only able to join in through Skype.  One family in particular always participates through Skype.  We do our best to make them part of the celebrations by strategically placing the computer monitor in the living room so they feel that they are actually there with us.  In fact, they have taken part in the “Secret Santa” gift exchange several times (with a little help from their friends). 

However, as invariably happens, the party migrates around the house, usually ending up in the kitchen. So, while they can take part to a certain extent, they miss out because of their lack of mobility.

The Solution

The defining objective of this project is to move the Skype session to a tablet and then mount that tablet on a robot that our guests can control remotely in order to allow them to join the party. We also aim to do this for much lower than $6,000 (the average price for a commercial telepresence robot).

While the defining objective provides a basis for design requirements,  it is really only a side-show.  The intangible (but far more interesting) objectives of this project are:

  • To learn about the Raspberry Pi; and
  • To spend some quality time with the kids doing purely geeky stuff.

The Team

Dad

Public servant by day, builder of gizmos by night.

Jessica

Chief architect.  Also in charge of aesthetic design.

Timothy

Mad scientist by day. Chaos monkey by night.

Overview

Figure 1 illustrates a rough outline of our intended solution.  The robot will consist of a basic robot platform, a power source, and a mount for a tablet. We decided that the mount will not be telescoping at this time in order to avoid having our robot tip over.  

Figure 1:  Telepresence Robot Concept

The final project deliverable is shown in Figure 2.  It consists of a mobile robot base with two drive wheels and a castor (not shown) mounted into an 8 inch square PVC container.  Inside the container, there is a controller board for integrating sensors and for providing control signals to a motor drive circuit.  There is also a single 12V 4 A/h battery.

On the top of the robot, there are two power switches (one operational, the other for future use), the Raspberry Pi itself, and a plexiglass plate mounted on a door hinge.  The plate is used to mount a tablet of any type to the robot using Velcro (two Velcro strips are shown).  The plate is hinged to allow it to tilt backwards in order for the tablet to point upwards.  The amount of tilt is controlled by a servo motor with a push arm located behind the plate. The plate is prevented from falling forward onto the Pi by using a pair of restraining cables attached to the corners.

The telepresence robot also has four Sharp distance sensors attached to the sides but these are not operational.  It was determined during testing that the distance sensors didn’t provide any more navigation information than what could be observed using the camera on the tablet.

Figure 2:  The Final Product

High-Level Design

Figure 3 shows the conceptual design for our robot. The system consists of two components - a remote control element running on the remote user's device, and the telepresence robot itself. These two elements will communicate over the Internet. We anticipate that this may be a challenge since both ends of the communication will be behind NATting firewalls (we'll cross that bridge later).

Figure 3:  High-Level Design

Locally, the telepresence robot will have a WLAN capability (provided by the Raspberry PI) and a Python Driver. The WLAN function provides a wireless capability while the Python driver acts as a communications interface between the Internet and the robot hardware and sensors (the GP Control board in the figure).

Instructions from the remote control (in Calgary) will be handled by the WLAN function and passed to the he Python driver.  The driver will interpret the remote control instructions and send them to the GP Control Board using the Serial Peripheral Interface (SPI) module available on the Raspberry Pi's I/O extension header.  Other options exist for communicating downward (e.g. RS-232).  However, I have only rarely used the SPI in the past and look forward to trying it again.

The robot hardware (GP Control Board, Robot Frame - essentially the bottom half of the robot) consist of a microcontroller, a motor driver board, and some sensor inputs.  Most microcontrollers are capable of communicating using the SPI protocol so the specific choice of microcontroller is not tightly constrained.

Mechanical and Electrical Design

As illustrated in Figure 4, this section describes the design of the robot frame and the GP Control Board that is used to drive the robot motors and collect sensor data.

Figure 4:  Focus - GP Control Board and Frame

Robot Frame

To maximum extent possible, the robot frame should protect the electronics and power supply from the hands of inquisitive kids and the paws of annoying cats. If it falls down a set of stairs, lead-acid batteries and little parts should not go flying everywhere.

But encasing the electronics, batteries, and motors presents an interesting challenge because most robot kits don't enclose their electronics and those that do cost more than I care to spend. So, my daughter and I went shopping for a low cost enclosure and we found the answer at Home Depot.

It turns out that an 8" by 8" PVC electrical box (shown in ) fits the bill almost perfectly. If more or less room is needed, bigger and smaller boxes are available.

Figure 5:  PVC Electrical Box

The reason this box makes an excellent choice is that, other than cutting holes in it to fit motors, switches, etc., no complex mounting considerations (and tools) are needed. It is as easy to work with as wood but looks quite a bit nicer in most cases.

The only disadvantage is that it is square. This could be challenging when rotating or moving the robot since the corners stick out in a way that a round base does not.

Motors and Wheels

The motors I used for this robot come from Princess Auto as a complete gearhead-wheel assembly as shown in Figure 6. I bought a pile of them in the surplus area of Princess Auto in 1996 and thought that was going to be the last of them. However, I just recently bought another pile from the same place. They seem to pop up sporadically in the surplus section from time to time. These motors are not terribly fast but they are solid and reliable.

Figure 6:  Motors and Wheels

Putting Motors into the Frame

The flat base of the motor fits nicely against the frame and allows it to be screwed in place in four locations. In order to attach the motors, I had to use a Dremel tool to shave down a small PVC column where the center screw goes on each side. After that, I cut a pair of holes, put the motors through and attached them to the sides.

The Balance Wheel

At this point, my intent was to put a very small castor on the front and back and centered like I’ve done before.  This prevents the robot from tipping forward or backward but does not hider turning.

But, because of the geometry of the motors I was using, I couldn't find castors small enough to mount underneath the robot without risking the possibility of the drive motors being lifted from the floor.  This could occur on uneven surfaces or when transiting from one room to another over a transition border. This has happened before.

My daughter suggested attaching another PVC box to the back of the robot and putting a castor on that. If the drive motor battery is placed at a point behind the center of the drive motors, then the weight of the robot will always be on all three points. And, I could move the other PVC box up or down to match the size of any castor.

Now that’s “thinking outside the PVC box”!  Figure 7 shows this plan in action.

Figure 7:  Frame, Drive Motors, and Swivel Castor

GP Control Board

The drive electronics for the GP Control Board are centered around the M68MOD912C32 microcontroller package (which TechArts sells as the NanoCore12 series) shown in Figure 8. It is a great little "hobby" device. I say "hobby" because, back in the day, the HC11 (like many of its peers) came in a DIP format. This allowed for a wide range of uses because it could be inserted into a breadboard. However, the HC11 began showing up in PLCC package formats (which were a little more challenging to use with a breadboard). With the arrival of QFP and surface mount, things could be very challenging indeed. The NanoCore12 addressed this concern by putting the HC12 into various DIP formats and included the bare essentials required to operate (BDM port, crystal, level shifters, etc.).

Figure 8:  NanoCore12 Module

This device is great for small projects, but there is a GOTCHA that you should keep in mind when planning to use it.  The Freescale 9S12 microcontroller is quite powerful. However, the core comes in a number of surface mount packages each with a different total pin count. The NanoCore12 is based on the 48 pin Quad Flatpack (QFP) which means that not all of the functionality of the core is available at the I/O pins.  Taking this one step further, the NanoCore12 exposes only 32 pins (not including BDM) which further constraints the I/O surface.

One of the key functions required of any microcontroller is Pulse Width Modulation (see the Annex on this).  In the 9S12C32, most of Port P is dedicated to PWM but these pins are not routed to the I/O port.  There is a register configuration that allows you to map port P onto port T to make PWM accessible.  The configuration enables the multiplexer shown in Figure 9 to repurpose Port T.

Figure 9:  PWM GOTCHA

The DIP pinout for the NanoCore12 is shown in Figure 10. From this figure, the mapping between the identified pins and the outputs of the MCU is not clear. For example, I could not tell that PM5 is SCK or that I need to configure DDRM in order to set SCK for operation. To address that problem, I mapped the 48 pin QFP  to physical pins on the 32 pin DIP as shown in Figure 10

Figure 10:  NanoCore 9S12C32 Pinout

I have often been faced with interfacing the 9S12C32 to various connector types that weren't natively supported on the particular control board that I was using. As a result, I was forced to cut traces to free up pins that were dedicated to other functions and to find prototype room somewhere so that I could solder in an appropriate socket.

To address this challenge, the GP Controller board takes all 9S12C32 outputs to a two-row female header as shown in Figure 11. From there, I can install a diverse set of connector headers for the sensors and actuators that I want. I use the female header and some jumpers to select an appropriate male header for each C32 input or output.

Figure 11:  Basic Schematic of "Generic" CPU Board

The jumper selections shown above configure the system to allow distance sensing on AN0-AN3. Motor drive and direction are provided by PT0-PT4 (where PT0 and PT1 are PWM and PT2, PT3 are configured as digital outputs).

Drive Electronics

The drive electronics for this frame are provided by a modified Lynxmotion L298 Dual H-Bridge. The schematic for my H-Bridge device is no longer available (I bought it in 2000). However, Version 2.0 of this device can still be found. The schematic in Figure 12 shows the V2.0 layout (mine differed only slightly).

Figure 12:  Drive Electronics

Figure 13:  Truth Table for One Motor

The truth table for the L298D (for one motor) is shown in Figure 13. In the drive circuit in Figure 12, we have wired A+ to A- through an inverter. This reduces the number of control signals from three to two. The signal that controls A+ and A- is now a direction signal andEnable provides speed control.  The drive circuit and control board are shown together during testing in Figure 14.

Figure 14: Drive Electronics Testing

Figure 15:  Drive Electronics in Frame

Figure 15 shows the drive electronics and GP Control board mounted in the frame.  It's a little crowded – next time we think we will go with the 12 inch electrical box.

Control System Design

Figure 23 identifies our current focus area in the overall project.  In this section, we will discuss the implementation of a Python driver on the Pi which will be responsible for proxying communications between the GP Control Board and the remote control.

However, before we were able to develop this material, we needed to do a complete reconnaissance of the Raspberry Pi. This is provided in Annex A.

Figure 23:  Focus - Python Driver on Raspberry Pi

Python Driver for the Raspberry Pi

There are two things that a driver on the Raspberry Pi must do. These are:

  1. Receive instructions over the Internet from a remote controller and possibly send data back (we'll leave the second part till later).
  2. Convey those instructions to the GP control board.

Therefore, we are going to need to define at least two communications protocols - one between the controller and Pi and one between the Pi and the GP Control board.

1.  Receive instructions over the Internet from a remote controller

In order to receive instructions from a remote control, I wrote a short Python script on the Pi that implements a server to listen on a socket and react to the instructions. The source that implements the socket server is shown below. This is prototype code and it certainly isn't robust.

I changed the rc.init script on the Pi in order to make this python script run at startup

2.  Communicating with the GP Control Board

I used the Serial Peripheral Interconnect (SPI) protocol to connect the Pi to the controller board.  This protocol was designed to allow devices on a circuit board (or within close proximity to each other) to communicate easily using very few wires.  In order to use SPI, a few things in Raspian needed to be adjusted in order for the kernel drivers to be available.

In a terminal window in the Pi, I edited the raspi-blacklist.conf file in /etc/modprobe.d andcommented (insert '#') on the line 'blacklist spi-bmc2708'. I hadto do this as superuser. One way to do this is as follows:

Once that was complete, I needed to get the kernel module that allows Python to interact with the BCM SPI pins. I executed the following commands at the command prompt on the Pi:

With all of this complete, I was able to insert the following import into my Pi code in order to access the SPI:

Googling spidev gave me the rather short documentation for the module. There are no configuration options that need to be changed in order to make it work "out of the box" with the SPI on the Nanocore (e.g. max_speed_hz, lsbfirst, etc). The only commands that need attention are xfer and xfer2. xfer allows you to specify a period on the signaling of the chip select line. In the final design, I'm likely to assert CS low by tying it to ground so that using xfer is not needed.  [UPDATE:  this choice was a bad idea….  Use the CS line].

The code segment below expands the previous segment to allow communications down to the GP Controller board through the SPI. I used a few IF statements to demonstrate the use of the SPI xfer2 command. The code segment does not show a complete communications protocol.

SPI Driver on GP Control Board

This GP control board was coded using the Freescale CodeWarrior free download. There are a number of options that can be selected on startup. One of them is the Processor Expert. This is a very useful tool if you completely understand all of the modules that exist on the HCS12 but I wouldn't recommend using it until you've gone through a number of manual programming exercises.

Figures 16 to 22 show the startup options I used.  It is interesting to note that most Freescale devices now come with an embedded monitor program in FLASH in addition to the standard Background Debug Mode interface.  This means that you don’t need a BDM pod to download and debug – although that is the preferred option because there are no restrictions imposed by the BDM infrastructure.

Figure 16:  Device and Serial Monitor Selection

Figure 17:  Code Options

Figure 18:  Project File Selection

Figure 19:  Include Initialization Code

Figure 20:  Memory Model Selection

Figure 21:  Just Say No to Lint

Figure 22:  Target Selection

To receive commands, the 9S12C32 on the GP Control board needs a driver listening to the SPI for instructions (and later, for giving back sensor information).

As I noted earlier, the GP Control Board is based on a Freescale 9S12C32 CPU in a "Nanocore" device from Technological Arts. However, the control board could easily have been an Arduino or some other microcontroller.

The Serial Peripheral Interconnect (SPI) protocol requires a master device and one or more slave devices. In our case, the master will be the Raspberry Pi and the only slave will be the NanoCore12.

The figure below shows the general case of an SPI master and SPI slave connected to each other. There are four signals (in addition to a ground wire) that need to be considered. These are SCLK (serial clock), MOSI (master out, slave in), MISO (master in, slave out), and SS* (slave select which is active low).

http://en.wikipedia.org/wiki/Serial_Peripheral_Interface_Bus

Figure: SPI Master-Slave (Wikipedia)

The clock signal is generated by the SPI master. When the clock begins signaling, a word (a byte, 10 bits, configurable) is transferred from the master to the slave over the MOSI line. At the same time, a word is transferred from the slave to the master over the MISO line.

So what happens if the slave has nothing to send? It just sends whatever is in the buffer for the MISO line. It doesn't matter because the master is not likely to use it anyway.

Another consideration: what happens if the master has nothing to send but the slave does? This requires the slave to either wait until the master is ready to push down a word (possibly buffering data) or the master must be able to send potentially meaningless data to the slave in order to collect the data that the slave has. In either case, both the master and the slave have to understand when garbage is coming over the line.

Finally, there is the issue of selecting a target slave device to talk to. This is the purpose of SS*. There is only one SS* line and it is active low which means that it is normally at +5V except when the master wants to select a device to talk to. In that case, the master makes the SS* line 0V and any device connected to the SS* and sees it go to 0V understands that the master wants to talk to it.

You might have noticed that there is only one SS* line which essentially means that only one device can be selected as a slave. The figure shown is the general case. Some SPI interfaced will have multiple SS* lines. For example, the Pi exports two SS* lines (CE0, CE1) on the I/O header. This allows the PI to address up to four slave devices.

In this project, I do not use SS*. I essentially assume that our slave device (the 9S12) is the only device around so we connect its SS* wire to ground (0V) which means that it is ALWAYS selected. [UPDATE:  Again, this was a bad choice.  Don’t do it].

Physical Interfacing

The figure below shows the innards of the 9S12C32 with focus on the SPI interface (pins 13-16 on the Nanocore's 32 Pin DIP).

Figure 24:  SPI Pins on 9S12C32

The SPI pins for the Pi are identified in the figure below. Note that I only connected pins 13, 14, and 16 from the 9S12. I wired pin 15 to ground (meaning that the 9S12 is always "enabled"). As a result, I did not need to connect pins 24 or 26 in the figure below. [UPDATE:  For the last time, this was a bad idea.  Don’t.).

Image:  http://www.raspberrypi.org/

Figure 25:  Raspberry Pi Pinout with SPI Interface Highlighted (Image:  http://www.raspberrypi.org/)

I wrote the code shown below for the 9S12C32 to process inputs from SPI and reflect them in the first nibble on Port T (PTT). This code was created using Codewarrior with startup scripts provided. I placed a series of LEDs on Port T so that I could confirm that the correct nibble was sent.

Here's a picture of my test harness in action. The point of this harness was to validate correct signaling between the Pi and the 9S12. The figure below shows a 9S12C128 variant rather than the Nanocore device. I used this during the interface testing because it had many more output ports.

Figure 26:  Test Harness in Action (Using 9S12C128)

After some initial bumps (I forgot to tie the CS signal on the 9S12 to ground), I managed to get any byte sent from the Pi over the SPI channel to show up on the T port through the LEDs. The next step was to port the code to the Nanocore12 and connect up some motors.

The code below is for driving the motors in response to commands from the SPI on the Raspberry Pi. Note that I made up the command protocol on-the-fly. I used powers of two for each command since that creates a minimum hamming distance of 2 (chances of noise transforming a command is therefore pretty small). Note that the code shown below is a kludge and is far from elegant. However, it is immensely readable which is more important at this point.

Python Driver on Remote Device

Figure 27 identifies our current focus area in the overall project.  In this section, we will discuss the implementation of a Python driver on a remote device which will give instructions to the robot.

Figure 27:  Focus Area - Python Controller on Remote Device

For this task, I've decided to write a short Python GUI with direction buttons as a starting point. However, there is no reason that the remote computer needs to run Python. Any scripting or programming language can be used as long as it can issue commands over the Internet.

I developed a prototype controller in Python to implement the following commands:

  • GoForward;
  • GoBackward;
  • TurnLeft;
  • TurnRight;
  • Stop;

The prototype controller code produces the window (using Tkinter) shown below.

Figure 28:  Basic Control Panel

The prototype is nothing to write home about (it is going to have to be much more robust and also allow two way communication) but it does the job for now.  The code behind the controller is shown below. Note that I have fed the IP of the controller into the code. It works on a contained private net but will not work over a double NAT. Also, the code is in gross need of optimization.

Integration

Figure 29 shows the GP Robot frame with the Raspberry Pi mounted to the top. Also on top are a pair of power switches and a piece of plexiglass mounted on a hinge.

Figure 29:  Integrated Robot

My daughter suggested applying a pair of Velcro pads to a piece of wood to hold the tablet in place. Matching pads will be applied to the tablet.

During our testing, we had the tablet taped to a piece of cardboard that pointed up at an angle. We discovered that, while looking up is good for chatting with people that are standing, it is really, really bad for driving (especially around stairs).

So, barely visible behind the plexiglass plate, we attached a servo motor with an arm that can push against the plexiglass. This will allow the plexiglass plate to be tilted backwards so that people on the other end of the video link can look up. But it's also very helpful when navigating to be able to look forward so the servo will allow the selection of the appropriate position.

The plexiglass tilt plate will be described in detail shortly.

Also shown in Figure 29 are a set of Sharp distance sensors bolted to the sides of the robot.  These are for later use and haven’t yet been connected.

Power

You'll notice in Figure 29 that there is a USB cable dangling from the Pi. This is a bit of a relic of trying to get power to all of the devices in the box. Here's a short history:

  • I tried using a 12V battery regulated down to 5V to feed the Pi and the sensori-motor interface (the 9S12) but the heat thrown off by the voltage regulator (even with a big heat sink) was significant. This state of affairs is due to the current draw by the Pi. The larger the current draw, greater the amount of power consumed in the drop from 12V to 5V.
  • My first compromise was to put two batteries inside the frame - one 12V (for the motors) and one 6V (for the electronics and Pi). This time, the heat wasn't too bad. Unfortunately, there was some unruly behaviour during testing which I blamed on the voltage regulator - my guess isthat the 7805 couldn't source enough current to run both the 9S12 (and sensors) and the Pi at the same time in a reliable fashion.
  • My second compromise was to buy a pair of 5V USB batteries and drive the Pi and the 9S12 separately (and get rid of the 6V battery). However, it turns out that the 9S12 doesn't draw enough current when the sensors are not in operation to prevent the 5V USB battery from going into shutdown. And besides, the USB batteries were less than an amp-hour and died at different times without much warning.
  • My third compromise was to replace the 6V battery for the 9S12, keep the 5V USB battery for the Pi (hence the dangling cable in the figure above), and keepa 12V battery for the motors. In short, three separate power supplies.

So, the third option was clearly not optimal. Each time we wanted to use the robot, we needed to connect three separate batteries and each would cycle down at different times.

To solve this problem, I ordered a Pololu D15V70F5S3 Step-Down Voltage Regulator (shown below) from Robotshop.ca. This device is a DC-DC voltage converter that doesn't spill a ton of heat going from 12V to 5V.

www.robotshop.ca

I hooked up the voltage regulator between the 12V supply and the robot drive electronics and the Pi and discovered, to my absolute amazement, that it didn't work. In order to be able to reliably send signals from the computer or mobile and have them executed by the robot, the Pi had to be powered by a USB battery rather than from the 12V battery.

Now, this made no sense which is why it was such an interesting problem. After some investigation, it turns out that hard wiring the Chip Select signal to "always on" is not a good idea in an electrically noisy environment. My logic captures showed that there were significant spurious signals happening on the SPI bus and that any time this occurred with the clock, the drive circuitry thought a message was coming down the pipe.

To cut down on this significantly, I modified the Chip Select signal for the drive circuit to feed directly from CE0 on the Pi.

Adding a Tilt Panel

Problems Seeing the World

In order to navigate, the tablet on the robot has to be facing horizontally - that is, directly to its front. However, in order to engage in conversation which someone nearby, the tablet must either be at face height (between 4-5 feet) or it must be able to tilt so that it can look up.

We already decided that the tablet would not be at face height. And, even if that was the case, the reverse problem would still be true: the tablet would need to be facing horizontally to converse but point slightly downwards in order to navigate.

To address this requirement, we placed a servo motor directly behind the plate holding the tablet as shown in Figure 30. On the servo, we attached an arm that would push the plate forward when required (so the tablet faces horizontally). Turning the servo motor would allow the plate to fall backwards to about 45 degrees.

Figure 30:  Servo Motor for Tilt Panel

Codingthe Servo and Drive Motors

As we mentioned earlier, the speed of the main drive motors are controlled using a Pulse Width Modulated (PWM) signal.  The position of the servo arm for the tilt plate is also controlled using PWM.  Both drive motors and servo motors therefore need PWM channels.

In PWM, we have a periodic signal (like a sine wave - but square...). The period is some arbitrary length of time. The signal repeats itself in each successive period.

Within each period, there is a certain length of time for which the signal is logic 0 (0V) and a certain length of time where the signal is logic 1 (+5V). The length of time for which the signal is logic 1 is called the "duty cycle" and it represents a percentage of the period. These concepts are shown in Figure 31

Figure 31:  Pulse Width Modulation

For the drive motors, a duty cycle of 100% (the signal is always at 5V) is interpreted as full speed. For the servo, the duty cycle determines where the servo arm sits in its rotation range.  Each type of servo tends to be different in this regard.

Configuring The 9S12 for PWM

Setting up PWM on the 9S12C32 has a few twists.  As I noted earlier, if you're using the 48 or 52 pin version of the C32, then you need to re-route PWM0-PWM4 from PortP P to Port T using the Mux box shown in Figure 32.

Figure 32:  Port P to T Mapping

This is done using the port T module routing register MODRR shown below.

Figure 33:  MODRR Register

Set the MODRR register bits to 1 to set the associated PWM to the port T pin (e.g., MODRR0=1 makes PT0=PWM0). Since we are going to use PW0 and PW1, and PW4, we set MODDR=0x13h. We use PW0 and PW1 for the motors and PW4 for the servo.

Next, we need to turn on PWM on for the selected pins. This is done through the PWME register as shown in the figure below.

Figure 34:  PWM Enable Register

We set PWME=0x13h to enable pulse width modulation on PT0 and PT1, and PT4.

The starting polarity (high or low) can be specified for each PWM channel using the PWMPOL register shown below.

Figure 35:  PWM Polarity

We will start the polarity for all PWM channels low so we set PWMPOL=0h.

The duty cycle of any PWM channel in the C32 is going to be based on some clock source and all clock signals are based on the bus clock. In the PWM module, two clocks, A and B, are derived from the bus clock by a pre-scale operation. Each of A and B can then be scaled again to provide two more clock sources - SA and SB (scaled A and B).  Each PWM channel can be set to use each of these four clock sources by the setting of the PWMCLK register shown below.

Figure 36:  PWM Clock Select

Finally, the period and duty cycle are controlled by the PWMPER and PWMDTY registers. To determine what values to stick in the PWMPER registers, we need to know a bit about what we plan to do with the PWM and what optimal frequency to set.

We're using a Hitec HS-325HB. The specs for this device can be found here. There are some electrical and mechanical specifications that should be consulted before selecting a servo for a specific job. These include:

  • Torque: The amount of force that the motor can apply when turning.
  • Speed: How fast the device will turn from one position to another.
  • Range: How much rotation the device can go through.
  • Voltage: What voltage the device operates from.

The first two are dependent to some extent on the voltage applied to the servo. In our case, I'm reasonably confident that there's enough torque to do the job, I'm not worried about speed, and I only really need one-quarter of a full rotation so range is not an issue (this servo has a 110 degree turning radius). In addition, I'm planning on operating it from 5V and this sits nicely in the applicable voltage range.

The period required for the HS-325 to work properly is approximately 20ms.  For a 20ms period, if the signal is logic 1 for 10ms (and logic 0 for the other 10ms), then the duty cycle is 50%.

The servo will turn to a spot (within its 110 degree turn radius) that depends on the duty cycle of the signal it is receiving. For this servo, what position corresponds to what duty cycle? 

By experiment, using clock source SA, I found that PWMDTY4=0x07 is approximately 0.9ms (the arm straight out so that the tablet looks horizontal to the ground) and PWMDTY4=0x0C is approximately 2ms (the arm in the center allowing the tablet to look up).

So, the final initialization routine for both drive motors and servo motor on the 9S12C32 is as follows:

In the Python module in the Pi, we needed to make a few code changes to reflect the new tilt functionality.  The code below shows the additional code.

Finally, in the Python remote control, we needed to add buttons for the tilt function.  We also decided to make the buttons a little bigger.  The kludge code for this is shown below.

The resulting control panel is shown in

Figure 37:  Final Controller Panel

Annex A: A Brief Reconnaissance of the Raspberry Pi

Introduction

This section provides an overview of the Raspberry Pi hardware and firmware.  The initial few sections don't assume any significant knowledge about hardware or software. They were essentially just my personal notes on figuring out what device I was looking at. However, the sections will become increasingly more technical as we progress but I will endeavor to explain things as I go.

Note: This annex is dated and reflects the very first model of Pi that came out. Many of the statements here may not reflect later models.

Model/Revision

"What Model/Revision am I working with?" It's always good to know what you're holding in your hand. However, I found that it wasn't entirely clear from the markings on the board what model and/or revision it was. Indeed, there appears to be several types of "Revision" number. One has to do with the PCB specs, there's one that seems to identify the manufacturer, and there's one that is related to the CPU. Fortunately, other people have had some issues with figuring out their hardware and they have been kind enough to document their findings in a number of places. One of these places is elinux.org which has three pages that are very helpful - one that gives a complete hardware breakdown, one that deals with the board history, andone that provides other specific hardware details. First, let's take a hardware tour of the device. The figure below shows the non-obvious parts of the Pi.

Figure 38:  Raspberry Pi Quick Overview

My board has the following devices:

Memory: Samsung 313 K4P4G324EB-AGC1 (512MB RAM chip) USB Chip: SMSC LAN9513-JZX RG2: 17-336 (RQB44) C6: 220 CFP 20N

I couldn't find an exact match on the hardware details page. I noticed that some of the numbers had "8" where there should be "B". Meh.

To find the board revision, the board history page suggests using a command-line prompt to query cpuinfo. This assumes that you have been able to get the Pi up and running with an operating system. If you haven't, there is plenty of info on getting this done all over the web. I would suggest using either of the following:

Note that when you first bootfrom the flash card, it will ask which distro you want to install. I Googled each to see what they were about although there is a short description attached at the NOOBS site.

Once you have Raspbian up and running, you can "open a prompt" and find out which board revision you have. Double-click on the LXTerminal icon (shown in the figure below circled in white).

Figure 39:  LX Terminal on Pi


You will see a new window similar to the one shown below.

Figure 40:  LX Terminal Open

Now you can use the keyboard to enter a command to report information about your Pi. Type in the instruction shown below:

The command produces a list of information as shown in the figure below.

Figure 41:  CPU Info

The important bits are Hardware and Revision. The information shown above is not representative of a real board because the screen-captures are from a Pi simulator in Qemu. However, my board reports the following:


Therefore, according to the table above, I am working with a Model B, Revision 2.0 board made by Sony (with 512MB RAM).

A Closer Look at the Pi Hardware

Because the Pi has a large following, there is plenty of good information available on many websites to help us out. This is particularly of value given that there are sections of the Pi that are not available for us to scrutinize. These are proprietary and, to know about them, you need to sign a Non-Disclosure Agreement. That's unfortunate.

In conducting this closer look, I have searched far and wide on the Internet. I am in debt to a number of individuals who have put their explorations out there for me to stumble upon. These include:

  • David Welch (dwelch@dwelch.com): https://github.com/dwelch67/raspberrypi;
  • http://elinux.org/RPi_Low-level_peripherals - This is the defacto location for everything low-level.
  • http://en.wikipedia.org/wiki/Raspberry_Pi
  • http://jeffskinnerbox.wordpress.com/ 2012/12/05/raspberry-pi-serial-communication/raspberry-pi-rev-1-gpio-pin-out/

The BMC2835 System on a Chip (SoC)

The core of the Raspberry Pi is the Broadcom 2835 System-on-a-Chip. If you look carefully for it on the board, you might be surprised to find that you can't find it. The figure below (from wikipedia) shows the SoC chip squarely in the middle of the board but if you inspect your board, you'll see that the chip there is a Samsung memory device. So what's going on?

Figure 42:  :  RaspBerry Pi Board (Courtesy: http://en.wikipedia.org/wiki/File:Raspberrypi_pcb_overview_v04.svg)

The memory chip is mounted on top of the SoC chip as shown in the figure below. This is a common practice for mobile devices where memory is concerned.

Figure 43:  Ball Grid Arry Mounting of Memory

The memory chip is put on top of the SoC using a Ball Grid Array (BGA). Wikipedia has a pretty good description of what that means electrically and mechanically. For us, the only thing that matters is that we now know where the CPU is located on the board.

I did quite a bit of digging to find out the hardware details of this device. However, it appears that SoC vendors live in a very competitive world and seem reluctant to give away details about the innards of their chips. Visit a number of locations around the web while searching for Broadcom specs and you'll see what I mean. Of course, that just makes the task of working at the hardware layer that much more challenging (in a positive way). [UPDATE:  Broadcom has since made their innards open source under a 3-clause BSD license http://www.raspberrypi.org/archives/6299) .

According to the Broadcom website:

The BCM2835 is a cost-optimized, full HD, multimedia applications processor for advanced mobile and embedded applications that require the highest levels of multimedia performance. Designed and optimized for power efficiency, BCM2835 uses Broadcom's VideoCore® IV technology to enable applications in media playback, imaging, camcorder, streaming media, graphics and 3D gaming.


What that means is that our little Raspberry Pi is optimized for multimedia.

Using the available sources from around the web, here is what I know of the Broadcom SoC:

  • The device contains two main processing units: an ARM1176JZF-S processor and Broadcom VideoCore Graphics Processing Unit (GPU).
  • My device contains 512 MB of DRAM.
  • The BCM2853 contains a number of peripherals as indicated in an interface manual [BCM1]. These peripherals are accessible to the ARM processor and include:
    • Timers
    • Interrupt controller
    • General Purpose Input/Output (GPIO)
    • Universal Serial Bus (USB)
    • I2C
    • Direct Memory Access (DMA) controller
    • Inter-Integrated Circuit (I2C) master
    • Serial Perphipheral Interfaces
    • Pulse Width Modulation (PWM)
    • Universal Asynchronous Receiver/Transmitter

For chipheads, schematics are usually the most desirable thing to have since they represent ground truth. Schematics are available from the Pi website. However, because schematics can be a bit daunting at first, I present aless detailed block diagram of the Pi in Figure 44 showing essential features within the SoC connected to devices on the Pi board.

Figure 44:  Broadcom SoC Block Diagram

BCM2853 PeripheralsOverview

The GPIO header connectors P1 and P5 give us access to some of the peripherals on the SoC. However, as illustrated in Figure 44, some of those GPIO pins are "dual purpose".  Figure 45 shows how each pin can be either a GPIO or can belong to one of the special peripherals of the SoC (i.e. I2C, UART). The actual function of each GPIO pin is configured by the software we write after the SoC boots up.

Figure 45:  I/O Configuration

Ground and Power Pins

Several pins are marked "Ground", "5V Power", and "3V3 Power". These pins, as you might have guessed, provide power and ground reference signals. If you connect any sort of circuit to any of the pins on the GPIO header, the ground reference of your circuit must be connected to one of the ground pins on the GPIO header. If you do not do this, the circuit you attach may not understand signal voltages the same way that the Pi does.

GPIO Pins

A General Purpose Input Output (GPIO) pin is an electrical interface that can be configured to do a number of different things. It can act as a digital input (i.e. accepts signals that are either 0V or 5V) or a digital output. They can also be configured to act as analog inputs and outputs (they can interpret and convert voltages on the pin between 0V and 5V into numbers and vice-versa). They can also be configured to do special functions such as being part of a serial port. We shall see some of these later.

According to the BCM 2835 ARM Peripherals manual, there are 54 general purpose I/O pins. Configuring each pin to do different things is achieved by writing data into one or more of the GPI0 configuration registers. There are 41 of these configuration registers as outlined in Chapter 6 of the Peripherals manual [BCM1]. We shall briefly cover a few of these registers below.

General Purpose Function Select (GPFSEL) Registers

There are 6 registers named GPFNSEL0 - GPFNSEL5. Each of these thirty-two bit registers is divided up into 10 groups of three bits (the final two bits are not used). Each three-bit-group determines what each respective pin does as follows:

Bit Pattern

Pin Function

000

The pin is an input

001

The pin is an output

010

The pin does alternate function 0

011

The pin does alternate function 1

100

The pin does alternate function 2

101

The pin does alternate function 3

110

The pin does alternate function 4

111

The pin does alternate function 5

Example: GPFNSEL0 is used to define what the first 10 GPIO pins do (GPIO0 - GPIO9) as shown in Figure 46.

Figure 46:  GPFNSEL0 Register

The "alternate functions" that are possible are defined in Chapter 6.2 of [BCM1]. These are shown in Figure 47. Every pin on each of the P1 and P5 headers is capable of alternate functions but most pins are not capable of doing every alternate function. If you plan on using an alternate function, it is important to know whether all of the pins that are necessary are accessible on the headers. For example, all but one of the pins for SD1 (GPIO 22-27) are available on the P1 header (GPIO 26 is not available).

Figure 47:  Alternate Function Mapping

General Purpose Output Set n (GPSET) and Clear n (GPCLR) Registers

If you define a GPIO to be a digital output pin, then you need a way of changing the value of the output pin from logic 0 (zero volts on the pin) to logic 1 (5 volts on the pin). In many types of CPU, this can be done by writing a 0 or a 1 to a register the controls the GPIO and the output pin will just mirror the value found in the register.

>

With the Raspberry Pi, this does not appear to be the case. Rather, setting a GPIO to output a logic 1 (5 volts) is done by writing to one register while setting the same GPIO to output a logic 0 (0 volts) is done by writing to a different register.

The registers GPSET0 and GPSET1 are used to set a GPIO to logic 1 (5 volts output). Since each register is 32 bits and there are 54 GPIO, a minimum of two registers are needed in order to cover every GPIO.

Figure 48:  GPSET and GPCLR Registers

There is a second pair of register GPCLR0 and GPCLR1, set up in exactly the same way, that are used to clear (set to logic 0) each GPIO.

Example Let's say we want to set GPIO 14 to logic 1, wait a bit, and then clear it to logic 0. First, we would use a bit-set instruction to cause the bit corresponding to GPIO 14 in the GPSET0 register to be set to 1.

Then, we wait.... Later we would use a bit-set instruction to cause bit 14 of GPCLR to be set to 1. Note how, in the second operation, we didn't write a zero to the register but rather a 1. Putting a 1 in the GPCLR register causes the associated bit to clear.

General Purpose Pin Level (GPLEV) Registers

Regardless of whether the GPIO is an input or an output, the two registers GPLEV0 and GPLEV1 can be used to read determine what state they are in (that is, logic 0 or logic 1). Once again, each bit corresponds to an individual GPIO so that a minimum of two registers is needed to cover all GPIOs.

General Purpose Event Detected Status (GPEDS) Registers

The registers GPEDS0 and GPEDS1 can be read to determine if a predefined event has occured on a GPIO. A zero indicates that nothing has happened and 1 indicates that an event has occured. Once again, each bit corresponds to an individual GPIO so that a minimum of two registers is needed to cover all GPIOs.

There are two types of event:

  • The signal on a GPIO goes from 0 to 1 or from 1 to 0 (an edge transition).
  • The level on a GPIO is detected to be a zero or a 1.

The type of event that is associated with each GPIO is configured with registers that will be discussed shortly.

General Purpose Rising Edge Detect Enable (GPREN) Registers

There are two General Purpose Rising Edge Detect Enable registers - GPREN0 and GPREN1. Each bit corresponds to one of the 54 GPIOs.

Let's say you attachGPIO 5 to a signal on an external circuit and you want to be notified whenever the signal goes from low to high (0 to 1, zero volts to five volts). You would set bit 5 of GPREN0 to 1 and then watch bit 5 of GPEDS0. Whenever that bit becomes 1, then you know that the signal has transitioned.

Other GPIO Detect Function Enable Registers

There are a pair of registers associated with each of thefollowing events that can be detected on a GPIO:

  • Falling Edge Detect Enable (GPFEN);
  • High Detect Enable (GPHEN);
  • Low Detect Enable (GPLEN);
  • Asyncrhonous Rising Edge Detect Enable (GPAREN); and
  • Asynchronous Falling Edge Detect Enable (GPAFEN).

The asynchronous edge detection functions are different from the rising and falling edge detection in that they are not tied to the system clock. That is, the rising and falling edge detection is performed at the beginning of specific clock cycles whereas the asynchronous detections occur regardless of the clock.

GPIO Pull-Up/Down

A pull-up or pull-down resistor can be added to an output pin in order to force it into either a high (5V) or low (0V) state when it is not otherwise being pulled into either of those states by an external circuit.

GPIO Communications Protocols

As shown in Figure 45, several GPIO pins can be grouped together and assigned alternate functions. All of these functions are essentially communications protocols. I describe each of them very briefly below. They are each fairly well covered on Wikipedia.

I2C - Inter-Integrated Circuit bus: A two wire computer bus protocol specified originally by Phillips for low speed communications between devices on a circuit board or between circuits that are close to each other. I2C requires two bidirectional wires - one for clock synchronization (SCL) and one for serial data movement (SDA).

SPI - Serial Peripheral Interface bus: This is a four wire serial bus that includes a synchronization clock signal (SCLK), two data exchange wires (MOSI and MISO) and a device select signal (SS).

>UART - Universal Asynchronous Receiver Transmitter: A UART implements the serial protocol that used to be associated with the serial port on the back of older computers. Most of the computers shipping nowadays have neither parallel nor serial port connectors anymore. The key difference between a UART and SPI, I2C, and USB is that there is no shared clock signal (the interface is asynchronous).

GPIO Register MAP

It's not totally straightforward... What is the specific address to be used when accessing a register? Pages 5 and 6 of the ARM Peripheral Reference Manual are somewhat confusing. However, on page 6, it does say the following:

“Physical addresses range from 0x20000000 to 0x20FFFFFF for peripherals. The bus addresses for peripherals are set up to map onto the peripheral bus address range starting at 0x7E000000. Thus a peripheral advertised here at bus address 0x7Ennnnnn is available at physical address 0x20nnnnnn.”

Therefore, to arrive at the location in memory to be used during assembly language programming, take the address presented as part of the manual, remove 7E and insert 20.

What is often done in code segments is to define the base address for all peripherals and then define base increments for each type of peripheral. Thus, we can define:

BCM_Peripheral_Base = 0x20000000
GPIO_Base = BCM_Peripheral_Base + 0x200000
PWM_Base = BCM_Peripheral_Base + 0x20C000
CLOCK_Base = BCM_Peripheral_Base + 0x101000
....

The base address for each of the other peripherals listed above can be found by scouring the Peripheral Reference Manual.  The full memory map for the GPIO is found on page 90 of the reference manual.

The Pi Boot Process

In this section, I take a closer look at the boot process.

The Short Answer

In order to have the Pi execute your code at boot time, the only thing you must really do is ensure that the file kernel.img exists on the FAT16 partition of the SDCard. In addition, that image file must begin at address 8000h. That's about it.

The Long Answer

The Broadcom chip is a proprietary affair. Unlike other CPUs, the ARM is not at the center of the boot process when power is turned on. Rather, it is the VideoCore GPU.

There are a number of good resources that explain the detailed steps in going from power-on to executing code at address 8000h by the ARM. Indeed, one of the easiest to follow is a video by Steve Halladay.

Essentially, the Pi boot process goes as follows:

  1. When power is applied, the GPU comes on and suppresses the ARM processor by putting into a reset state.
  2. A section of ROM in the GPU loads the file bootcode.bin into L2 cache and begins execution.
  3. This code enables the Pi's SDRAM and then loads start.elf (the GPU's firmware) into it.
  4. start.elf reads config.txt and cmdline.txt for configuration options.
  5. kernel.img into SDRAM.

Again, we note that bootcode.bin and start.elf are proprietary code.