The Nabaztag/tag is made by a company called Violet. It is Violet’s 2nd generation Nabaztag and adds a few new features on top of the 1st generation. There is a relatively strong Nabaztag community on the internet. Quite a few sites exists with neat software hacks and documentation that allow users to have some control of the device. However, I hadn’t seen anyone really take advantage of the hardware in any way other than what Nabaztag’s inventors intended. From what I gather, its possible to run bytecode on a “virtual machine” on the Nabaztag, but that doesn’t give you full access to the Nabaztag’s hardware.

It would be great to gain complete control of the Nabaztag – ie download a custom version of the firmware such that you would have complete control of the processor and peripherals. This would be a fun project, and might open up the possibility of adding additional sensors or output devices to the Nabaztag to expand its features. Additionally, it would open the door to using server implementations completely unassociated with Violet, which occasionally has “service problems”.

In a perfect world, the firmware for the Nabaztag would be stored in a nice, neat, extractable flash chip which could be easily removed, backed-up, replaced, modified, etc. Alas, Violet was not so considerate:) During my investigation I had to dismantle the Nabaztag. I snapped some pictures during this process:

“Important” Component Summary

OKI ML67Q4051 33MHz ARM7-TDMI processor

• 128 KB of “secure” internal flash
• 8KB “Boot Flash ROM”
• 16 KB internal RAM
• Supports 32-bit instructions or 16-bit thumb instructions
• JTAG interface
• External memory controller
• Peripherals – I2C, SPI, watchdog, RTC, timers, 4 ADCs, DMA controller, 2 UARTs

OKI ML60842 USB 2.0 OTG controller

• Used to interface ARM processor with Ralink 80211b/g module

Samsung K6X8016T3B 512K x 16 bit CMOS SRAM

• Additional 1MB of RAM available to ARM processor

(1 MB)VLSI VS1003B MP3/WMA/WAV/MIDI decoder, ADPCM encoder

• Play streamed audio, encode voice commands

Texas Instruments TLC5922 16-bit LED driver

• Drive belly button and nose LEDs
• Serial data interface, SPI compatible

Ralink RT2571W-based USB 802.11b/g module

• 802.11b/g adapter
• USB interface

STMicro CR14 contactless coupler with anti-collision, CRC management

• Used for RFID
• Supports ISO14443 type-B protocol
• I2C interface

Analysis

It looks like Violet did their homework and made it relatively hard to extract the firmware from the Nabaztag. The 128 KB of internal flash that is used to store the firmware has a security feature which will make it difficult to read. After Violet has programmed the firmware, they can put the enable a “security mode”. When this security mode is enabled, external devices (eg JTAG or parallel programmer) can no longer read or with the flash, or use the standard JTAG debug functionality of the chip. OKI describes this feature as follows:

When a security bit is set, the contents of Flash ROM cannot be read or rewritten in Flash-JTAG mode. A debugging interface (ie JTAG) can no longer be used. The security bit can be reset after the chip contents are erased in Flash-JTAG mode or parallel programmer mode.

So if one were willing to erase their processor’s flash, they could then reprogram it with their own image. This means the Nabaztag is pretty easily reprogrammable, but it would be pretty difficult to reprogram it with any sort of useful image since its not possible to inspect the current Nabaztag firmware. Without reverse engineering the current firmware, it would be VERY difficult to port/write some of the current Nabaztag features such as the wireless card driver.

Possible Workarounds

A few possible ways to read the flash on the ARM processor came to mind. All would require a significant amount of work (and maybe not be possible at all:).

Make Sure “Security Mode” is Enabled

I made the assumption that Violet enabled the security mode on the processor do prevent reverse engineering. Unfortunately I don’t have a JTAG debugger compatible with the L67Q4051 to test this. The first thing to do would be to verify that security mode is in fact enabled. If anyone has an ARM JTAG debugger lying around or wanted to donate one to the cause, I’d be happy to look into it:)

Use Serial Programming Interface

The ARM processor has some configuration pins that affect how it operates when it comes out of reset. One combination of these configuration pins forces the processor to execute the Boot ROM flash (the 8KB, non-user reprogrammable flash) and allows the user to erase and reprogram the 128 KB of flash by downloading an image over the serial port. You can do it with a standard terminal emulator such as TeraTerm or minicom or with OKI’s own utility called ISFP which is available their its website.

The ISFP utility can actually dump the flash for other OKI chips like the ML674001 and ML675001. However, OKI says this feature is not available on the L67Q4051 which the Nabaztag uses. Its not clear to me whether this feature is truly disabled on the L67Q4051, or if OKI’s ISFP software just doesn’t allow it. Its a bit risky, but it may be possible to issue the same serial commands to the L67Q4051 that are known to work on the ML674001/ML675001.

Use “User Mode” Programming interface to read the flash

Code executing on the L67Q4051 is able to read the 128 KB of flash even when the “security mode” is enabled. If the code executing on the L67Q4051 could be comprised, it may be possible to execute a program which would read the flash contents and print it to the serial port. I imagine comprimising the L67Q4051 would be no trivial matter…

Reverse Engineer Nabaztag Firmware Published by Violet

Violet provides a firmware image at www.nabaztag.com/firmware/firmware.0.0.0.10.sim.txt. I’ve looked at this file a bit, and its not clear to me what format it is in or how the Nabaztag firmware interprets it. Its possible that this file could be reverse engineered to provide a firmware image that could be inspected in order to make a custom image for the Nabaztag.

Reprogram “Fresh” Image

Documentation for most of the peripherals are readily available. Source code is also available for the more complex components such as the VS1003B and Ralink wireless adapter. In theory, it would be possible to write Nabaztag firmware from scratch and not use the stock Nabaztag firmware as a reference at all. However, this would be VERY time consuming, and frankly, I don’t have that much time on my hands:)

Conclusion

Reprogramming the Nabaztag is straightforward and could be done with little effort. However, first extracting the firmware image to reverse engineer would pose a significant risk to bricking the Nabaztag and could take significant effort. If others have any additional info or thoughts, please let me know!

1. It does basic macro expansion/replacement on html files. This allows html files to contain non-html tags which can be expanded into html based on the values in a configuration file. For instance, since every html file has the same header, its possible to include an <auto_update_header> tag in every html file and then have 1 definition of <auto_update_header> it in a configuration file. Its then easy to update the entire site’s html by modifying the configuration file and running update_html.py. This makes it easy to make sitewide changes and ensure all pages are up to date.

2. It “intelligently” uploads the website a webserver via FTP. It would be easier to use a standard FTP tool which could perform syncing, but there are many files that I don’t want to be uploaded (eg. .svn directories, non-gif or non-jpg images, etc). The FTP server address, username, and password as well as file and directories to be ignored can be specified in the same configuration file referenced above.
The script is very simple at this point, but should evolve as this webpage does.

Download update_html.py
Download update_html_example.conf
Download update_html_example.html

Download legofy.py
Download Lego Pattern
Download Lego Highlight Pattern

Introduction

During the cold winter months in Wisconsin, avid iPod users find difficulty in comfortably controlling their MP3 player. In order to manipulate the touch-based iPod controls or Apple’s standard remote control, pockets must be opened and gloves or mittens removed, losing precious heat to the cold. One possible solution would be to use general body motion to control the iPod as opposed to touch. This would require no dexterous tasks, only general movements

This same concept would also make controlling an iPod easier for active people (eg. riding a bike – you can keep your eyes on the road and don’t have to use fingers) and those who lack dexterity and can’t use their fingers to manipulate the iPod controls (eg. arthritis sufferers).
Summary

The goal of the motion-based iPod remote control was to design and build a remote control for the iPod that responds to a user’s motion. An accelerometer is used to detect the motion of a user. A microprocessor then interprets these movements and sends the appropriate commands to the iPod via its remote control port.

Hardware Components

• iPod: If you’re here, you probably know what one is. Check them out here if not: www.apple.com/ipod. This project is tailored to the 3rd generation iPod (and should be compatible with 4th, U2, photo, mini revs too), but can be adapted to work with any generation.

• Accelerometer: A Memsic MXD2125 accelerometer was used to sense the user’s motion. It is a two dimensional accelerometer and senses acceleration in the range of -2g to 2g. It uses pulse width modulation (PWM) to output acceleration measurements. It measures both static and dynamic acceleration. Internally, it heats up a bubble of air and uses an array of thermocouples to detect its position and thus acceleration.

• Microcontroller: An ATmega8 microcontroller was chosen. It is an 8-bit AVR from Atmel. It polls the accelerometer for data, interprets the readings, and sends commands to the iPod. The ATmega8 was a bit overkill but had all the necessary features (Interrupts on IO pins, onboard UART, internal oscillator, 16-bit timer, etc) and also enough flash and memory to expand the remote in the future. An ATtiny2313 was also used during the development process.

• RS232 Line Level Converter: A Maxim MAX3233E RS232 line-level converter was used during development to translate the serial output of the microcontroller to RS232 voltage levels needed by a PC’s serial port. Internal charge pumps are used in the MAX3233 which reduces necessary external circuitry.

iPod Remote Port

The 3rd generation iPod has a remote connector on its top next to the headphone jack.

The remote connector has 4 pins- VCC, GND, TX data, and RX data. The remote connector supplies power to the remote control as well as receives its commands. For pinouts, check here.

Top of ipod 3rd Generation iPod viewed from above.

iPod Remote Protocol

In order for the ATmega8 to send commands to the iPod, it had to mimic the standard iPod remote control’s output. This reverse engineering of the iPod remote protocol can be done by hooking up an oscilloscope to the remote control Apple sells and monitoring its output when different buttons are pushed. Fortunately, this step had already been mostly done by someone else. The iPod remote uses a 7-byte standard 8N1 19200 baud serial protocol. The commands contain a checksum and you’ll also notice that the “command byte” (byte 6) using bitmapping so that multiple commands can be sent at once. The commands are defined as:

Initial Prototype

An initial prototype was created using a breadboard. It utilized an ATtiny2313 rather than the final ATmega8. Although the ATtiny2313 could have been used in the final implementation, its 2Kbyte flash limited its possibilities for growth. The accelerometer was soldered to a Memsic MXEB-002 eval board then connected to the breadboard via 4 wires. The accelerometer eval board (sans accelerometer) can be seen in the upper left of the image along with the rest of the prototype setup.

Prototype setup.

Schematic/PCB Creation

The prototype circuit was drawn up by hand and was relatively straight-forward. The next step was to create an electronic version of the schematic and its corresponding PCB. PowerLogic and PowerPCB were used to create the schematic and PCB, respectively. To reduce the final size of the remote itself, the MAX3233E RS232 converter and programming circuitry was placed on an external “debug board” which would connects via a flat flex cable to the “remote board” which has the accelerometer and microcontroller.

The schematic entry was straight-forward, however the PCB layout was not quite as simple. Decals had to be created for non-standard symbols, design rules checked against the PCB fabricator rules, etc. This may be easy for someone with extensive experience, but was a new experience for me and took a good deal of time. PowerLogic and PowerPCB are designed to work together and thus when the schematic would change, the change could be “forwarded” to PowerPCB with little hassle. The process took a while, but was a good experience all-in-all. The final remote has 1 red power LED and 3 general purpose LEDs connected to some of the ATmega8′s spare IO pins. The packages were LLC8 for the accelerometer, TQFP32 for the ATmega8, and 603s for the discrete components.

Debug setup Debug board (left) <-> Remote board (right).

Debug board schematic Debug board schematic.

Remote schematic Remote board schematic.

Programming/Serial Debug

As mentioned above, the “debug board” contained the circuitry necessary to program the ATmega8 via a PC parallel port as well as convert its serial I/O to standard RS232 serial port voltage levels. The “debug board” connects to the parallel and serial port of a PC via a RJ45 jack which split into two connectors. RJ45 headers were used on all ends of the cable and the serial and parallel connector had female RJ45 connectors on them. Only 7 signals were needed, so standard ethernet cable and RJ45 headers worked great. I’ll draw up a wiring diagram for the Programming/Serial cable one of these days (it can be derived from the schematic if necessary).

To connect the remote control to the iPod, a connector was fashioned using an old ISA card and some 2-row header pins. Berg pin connectors were crimped onto the ends of the wire then insulated with heat shrink. Credit goes to the the iPod Linux project for the idea. Check out putting Linux on your iPod while you’re at it.

An sp12 programmer was used to program the ATmega8. It is a software/hardware combo that has programming software under both Linux and Windows. It uses a standard parallel port. The sp12 hardware is located on the “debug board”.

serial debug and programming cable Programming/Serial debug cable + headers.

ipod remote connector based on old ISA card ISA connector-based remote connector.

remote connector plugging into the ipod Connector + iPod.

Result

The final remote control was roughly the size of a quarter, slightly smaller than the standard Apple-supplied remote:)
finished remote Final remote.

Software Components

• Development Environment: avr-gcc was used to compile code for the ATmega8. avr-gcc allows a host computer (x86 in my case) to cross-compile for an AVR target using the standard GNU tools like make, objcopy, ld, etc. avr-gcc is supported under both Windows and Linux(which I’d recommend:). All code for the project was written in C.

• AVR Programmer: As mentioned above, a sp12 programmer was used to download the compiled files to the ATmega8 via a PC parallel port. The software to program it is available for both Windows and Linux.

Reading Accelerations

The MXD2125 accelerometer has two outputs – one for X accelerations, one for Y. These accelerations are represented by the duty cycle of a square wave. A 50% duty cycle is equivalent of 0g. Duty cycle’s above 50% are positive accelerations, duty cycles below 50% are negative accelerations. A formula provided by Memsic converts the duty cycle reading into acceleration. To read the accelerations, the INT0 and INT1 pins of the ATmega8 were connected to the MXD2125 outputs. These pins allow triggering of external interrupts. Additionally, they can be configured to trigger on positive or negative edges, or logic level changes. This allows the ATmega8 to read the duty cycle using the following process:

• Interrupt on a positive edge, record the time
• Reconfigure the pin to trigger on a negative edge
• Interrupt on negative edge, record time
• Disable interrupts on the pin
• Repeat the process on the other input, thus reading both accelerations

These times in addition to the constant period of the MXD2125 output can then be used to calculate the duty cycles, and thus the accelerations.

Interrupt-based duty cycle calculation.

Interpreting Accelerations

This was definitely the hardest part of the remote. The accelerometer detects both static and dynamic acceleration in 2 dimensions. Therefore, it can detect movement in a plane, or, using simple geometry, the angle of tilt of the sensor. The angle of tilt is measuring what portion of Earth’s gravity is acting horizontally on the sensor rather than acceleration caused by movement.

Since everyone moves at different rates while gravity is constant, it is much easier to accurately detect tilts. Thus the angle of tilt of the accelerometer was most often used in the algorithm to interpret commands. A state machine was then implemented in software which would interpret the user’s motions into commands. To prevent the remote from generating random commands while running, waving, etc, the remote must be put into a “command mode” by holding the remote flat for approximately 1 second before it begins issuing commands. Once the remote begins accepting commands, multiple commands can be issued until the remote is held still for approximately 1 more second.

The movements could be more complex, but for the purposes of a prototype, the above worked fine and demonstrated the remote’s functionality. In the movie section below, you can see a different detection algorithm that responds to circular motions to control volume.

Tilt calculation.

Command recognition stat machine.

Source Code

To be released…

Result

The final result was a success. The Motion-Based iPod remote control met the goals set out at the beginning of the semester. It was a lot of fun to design and it I got two independent study credits out of it to boot:) It won first prize for individuals at the University of Wisconsin Engineering Expo this fall as well. The final implementation was programmed to be worn on the user’s wrist, but could be reprogrammed to be worn on a headphone, in a pocket, etc. I’m currently pursuing the possibility of getting the remote or some of its technology patented via the Wisconsin Alumni Research Foundation.

Future Possibilities

• More complex motion detection
• Wireless
• 3D accelerometer
• TV/DVD/VCR/etc remote
• Audio feedback

Movies

This movie demonstrates the Play, Volume Down, Volume UP, Fast Forward, and Pause commands.

This movie is a close-up demonstration of a Pause, Play, Volume Down, Volume Up, Fast Forward, and Rewind commands. Notice the small amber LED on the remote. When lit, it signals that it is ready to accept commands. Another amber LED blinks when a command it sent giving visual feedback to the user.

This movie demonstrates an alternate detection algorithm to detect volume up and down commands. The remote is configured to interpret clockwise circular motions as “Volume Up” and counter-clockwise circular motions as “Volume Down”. It is meant to allow the user to mimic the scroll wheel finger motion with their arm. More creative patterns could be developed. Mr. Miyagi would be proud…

In the movie, the remote is shook (like it would be in a real world situation if it were strapped to your wrist) No commands are sent while it is being moved since it is in a “Wait” state. After it is held flat for 1 second, an LED lights up signaling it is ready to accept a command. Each time the remote is tilted upwards, the volume on the iPod increases and a LED on the remote flashes showing the user that a command has been sent. Tilting the remote downward would decrease the volume similarly. The remote is held flat for 1 second to put it back into the “Wait” mode.

In the movie, the remote is shook (like it would be in a real world situation if it were strapped to your wrist) No commands are sent while it is being moved since it is in a “Wait” state. After it is held flat for 1 second, an LED lights up signaling it is ready to accept a command. When the remote is moved quickly left to right, a Pause command is sent to the iPod and a LED on the remote flashes showing the user that a command has been sent. The above process is repeated to issue the Play command.

In the movie, the remote is shook (like it would be in a real world situation if it were strapped to your wrist) No commands are sent while it is being moved since it is in a “Wait” state. After it is held flat for 1 second, an LED lights up signaling it is ready to accept a command. Each time the remote is tilted to the right, the iPod skips the the next track and a LED on the remote flashes showing the user that a command has been sent. Each time the remote is tilted to the left, the iPod skips to the previous track and a LED on the remote flashes showing the user that a command has been sent. In the movie, the remote skips ahead two songs, then skips back one song.

Datasheets/Links

• MXD2125 2D Accelerometer
• Atmel ATmega8 Microcontroller
• Maxim MAX3233E RS232 Level Converter
• gcc + avrlibc
• iPod Remote Control Protocol
• sp12 AVR Programmer

Questions, Comments?

I probably didn’t do the best explaining certain aspects of the project, so if you have questions or comments, email me @ petertyser @ nospam (dot ) gmail (dot) com.

(Limited) Details

A Basic Stamp 2 was used in conjunction with ADCs and other external circuitry to record data. The data logger could be connected to a computer via a serial port. The host PC could then read the data from the Basic Stamp 2 and post it to the internet. The software running on the PC was written in Visual Basic (Uck…) and implemented a very trimmed down web server to post the results so that they could be viewed using a standard web browser.

Result

All in all, the project was relatively straightforward. It was a good intro to a simple embeded system and I got 3 credits from it, which I can’t complain about. The schematics and source code have been lost in the clutter of my room…