Sal's Hacker Space: April 2013

I recently attended the +Udacity HTML5 Game Development study group hosted by +Colt McAnlis , +Sean Bennett and +Peter Lubbers in San Francisco.

It was an awesome experience as there was a lot to learn not only about HTML5 but also about game development in general. Colts rants were very very instructional.

I met +Derek Duarte there and two of us immediately hit off. We started building a game for the HTML5 competition, but unfortunately due to work and personal reasons were unable to finish in time for the contest.

But all was not lost. We have decided to continue on with the game and eventually launch it on the web and Android.

The game is a coffee shop management game where you try to manage your inventory based on customer preferences. The goal is to make the most amount of money you can and build a coffee empire all over the city. (incidentally most of the dev work for this game happened in a coffee shop... how... quaint)

It is based on the popular Side Walk Empire comic strip made by +Eddie Ahn. You can follow his work here: http://www.ehacomics.com/

Here is a brief video of what the prototype currently looks like:

I am running this on the desktop with the help of the wonderful Libgdx by +Mario Zechner and his team of snazzy code monkeys.

A lot of work still needs to be done on this some of which is:

New Levels / Shops
Improved Animation
Optimization
....

...and the list goes on.

Dev work on this will continue and we will hope to see a finished product by the end of the year.

Props to Udacity and Libgdx that made this possible.

Note: I am aware of the resolution issues of the video on this page. They will be fixed as soon as possible.

If you search online for LCD RSS feed readers, youll notice that most of them target twitter. Wheres the love for Google+? Right here.

I wanted to make a display, which i could put on my desk, that would stream Google+ feeds. That way i could resist the urge to log onto Google+.

You will require the following for this project:

1) 1x Arduino Uno

2) 1x LCD Display. Anything compatible with the Hitachi HD44780 driver will work.

3) Some jumper wire.

4) Basic knowledge of python and HTML

5) Google Plus Developer Key.

To find out how to get your developer key go here: https://developers.google.com/+/api/oauth#apikey

NOTE: Depending on whether you have a perf-board or a bare Arduino shield such as the maker shield, you might or might not need a soldering iron and some hookup wire.

How it all works:

We will be using Python and the Google Plus API to pull activities that people that we want to stalk....er follow post. The Python script will store all activites onto a file and then read it line by line and send it in chunks to the Arduino through serial communication. The Arduino would then display the information on the LCD screen.

We could have sent directly without saving it to a file but during debugging i was getting some goofy characters on the console screen if i directly printed data to it. Oddly enough saving the information to a file and then reading from it fixed everything.

We will send the information in chunks of 64 char because that is the limit of the Arduino's Serial RX Buffer. Sending information in a continuous stream could result in losing data. The Arduino could be saving the information in the buffer to a char array and at the same time the python script could be sending the rest of the data through serial. That would be bad.

That being said, lets start building.

Connecting the LCD Screen to the Arduino:

This process is fairly painless. I followed the following tutorial:

http://arduino.cc/en/Tutorial/LiquidCrystal

I suggest you do the same. Once your setup come back.

Accessing Google+ using Python

Pulling information from Google+ is relatively simple. At the time of writing these API calls are read only.

You will need to install the PySerial module to be able to transmit over Serial.
This can be found here: http://pyserial.sourceforge.net/

The httplib2 library to create your http object found here: https://code.google.com/p/httplib2/

The Google+ API python starter found here: https://code.google.com/p/google-api-python-client/

Here is the code:

	import httplib2 #Module to create Http object
	import serial #Module for communicating through serial port
	import time #Module for adding time delay
	import math #Module to round floats up to ints.
	import random #Module to generate random numbers. These numbers randomly pick one of the prestored Google+ userIds to display

	from HTMLParser import HTMLParser #To parse the HTML data we will recieve from the API calls.
	from apiclient.discovery import build #To import build object to create Google+ service.

	class MLStripper(HTMLParser): #Class to strip HTML tags and special characters. Courtesy of the good folks at stackoverflow.
	def __init__(self):
	self.reset()
	self.fed = []
	def handle_data(self, d):
	self.fed.append(d)
	def handle_entityref(self, name):
	self.fed.append('&%s;' % name)
	def get_data(self):
	return ''.join(self.fed)

	def strip_tags(html): #Strip tags function
	s = MLStripper()
	s.feed(html)
	return s.get_data()

	def split(str, num): #Split function to split the information into user defined chunks.
	return [ str[start:start+num] for start in range(0, len(str), num) ]

	#Creating http and serial objects. The serial object sets up the port and baud rate.
	http = httplib2.Http()
	ser = serial.Serial("/dev/cu.usbmodem1411",9600)


	#An array of pages we want to display with their ID. Enter IDs of your own choice.
	feedsList = {'#AsapSCIENCE': 101786231119207015313,'#Life at Google':104898479113219628100, '#LifeHacker': 107345380056943591322 }
	feedNumber = random.randint(0 , 2)

	#Create the Google+ service using your developer key and randomly choose which feed to display
	service = build("plus", "v1", http = http, developerKey = "Your Developer Key goes here..")
	activities = service.activities().list(userId = feedsList.values()[feedNumber], collection = "public").execute()

	items = activities.get('items') #Get a list of the activities of the desired page.

	file = open('Feed', 'r+') #Create/Open file object
	file.truncate() #Clear all previous information
	for item in items: #Loop through the activities in the list
	content = item['actor']['displayName'] #Get items display name
	#content = content.encode('utf8')

	file.write("" + content + ": ") #Write the display name followed by:
	content = item['object']['content'] #Get the content.

	content = strip_tags(content) #Strip HTML tags from the content.
	file.write(content) #Write the content onto the file.
	file.write('\n') #Write a newline character.
	# ser.write(file.readline())
	file.close() #Close the file.
	print "file opened and closed successfully..."

	file = open('Test', 'r') #Open the file for transmission over serial port.

	print "opening file again to transmit..."

	for line in open('Test', 'r'): #Loop through each line in the file.
	ser.write('r') #Write a ready char to serial to let Arduino know data is incoming.
	reading = 'y' #Set Arduino reading state to yes

	newsFeed = file.readline() #Read new line from the file.
	length = newsFeed.__len__() #Get the length of the line.
	transNo = length/63.0 #No of times to transmit if data was sent in chunks of 63 char for each line.
	transNo = int(math.ceil(transNo)) #If recieved float, round it up.

	print transNo
	dataSlice = split(newsFeed, 63) #Split data into chunks of 64 characters
	print dataSlice
	if transNo > 6: #Only send a maximum of 6 transmissions
	for x in range(0,6):
	ser.write(dataSlice[x])
	print dataSlice[x]
	time.sleep(1)
	else:
	for x in range(0,transNo): #If less than 6 transmissions then send all of them.
	ser.write(dataSlice[x])
	print dataSlice[x]
	time.sleep(1)

	while reading != 'n': #Wait for Arduino to send command for n line before sending.
	#print "waiting for ready"
	try:
	reading = ser.read()
	except serial.serialutil.SerialException:
	pass
	file.close()

view raw googlePyReader hosted with ❤ by GitHub

The code is pretty self explanatory. Ive tried to comment it as best as i can.
Heres a brief summary of whats happening:
An http object is created to pull information of interest from Google+. Make sure you use your developer API key. A dictionary holds information of the pages to be displayed on the LCD screen. You can enter your own pages here. A random number is generated to pick a random index in the dictionary and pull that index's contents and display name and save them to a file. The strip function strips all HTML elements from the content.

Once information has been stored to the file, it is opened and is read back line by line (You dont need to close and open it like i did). Every time a line is read it is broken up into chunks of 64 characters. This is because the RX buffer of the Arduino can only hold 64 characters at a time. If data was sent in a continuous stream then the buffer would overflow and some data would be lost.

If the data is very large it will be transferred a maximum of 6 times. This is because of the small LCD i am using.

Once a line has been transmitted, the script will wait for the Arduino to request another line by sending the 'n' character.

Once all information has been read, the file is closed before the script is terminated. This script can be scheduled to run at specific intervals. I have set it up to run every 30mins.

Reading from Serial with the Arduino

Now that we are sending data through the serial port, lets read it with the Arduino.

Here is the code, Ill explain how it works below

	/*

	G+ Reader

	A scrolling Google+ display for Arduino
	By
	Hamza Salman Afzal.

	*/

	/*
	LiquidCrystal Library

	Library originally added 18 Apr 2008
	by David A. Mellis
	library modified 5 Jul 2009
	by Limor Fried (http://www.ladyada.net)
	example added 9 Jul 2009
	by Tom Igoe
	modified 8 Feb 2010
	by Tom Igoe

	This example code is in the public domain.

	http://www.arduino.cc/en/Tutorial/LiquidCrystal

	The circuit:
	* LCD RS pin to digital pin 12
	* LCD Enable pin to digital pin 11
	* LCD D4 pin to digital pin 5
	* LCD D5 pin to digital pin 4
	* LCD D6 pin to digital pin 3
	* LCD D7 pin to digital pin 2
	* 10K resistor:
	* ends to +5V and ground
	* wiper to LCD VO pin (pin 3)

	*/


	#include <LiquidCrystal.h>

	LiquidCrystal lcd(12, 11, 5, 4, 3, 2);

	int bytei; // Index for Serial Data to String
	int col; // Column variable for scrolling
	long startTime; // Read start time
	long loopTime; // Time spent reading
	char statusmsg = 'n'; // Arduino status message n = No, r = ready
	char msg[400]; // Message buffer
	unsigned long newLineTime;
	unsigned long retryTime;

	void setup() {
	lcd.begin(16, 2); // Set up the LCD's number of columns and rows:
	lcd.clear(); // Clear the LCD Screen
	lcd.noAutoscroll(); // No AutoScroll
	lcd.setCursor(3,0); // Set the cursor at position 3 to print Title Message
	lcd.print("G+ Reader!"); // Set title message
	lcd.setCursor(0,1); // Set cursor to lower row
	lcd.print(" (c) Sal 2013"); // Set Copyright Info
	lcd.setCursor(0,0); // Set cursor at top left corner of LCD screen
	Serial.begin(9600); // Set Baud Rate
	delay(1000); // time to start up script
	}

	void loop(){
	// Wait for read request. If read request recieved then start reading
	// and track time
	if(Serial.available()){
	statusmsg = Serial.read();
	if (statusmsg == 'r'){
	startTime = millis();
	}
	}

	retryTime = millis(); // Keeps track of how long no data has been transmitted

	// If no data has been transmitted for more than 10 seconds, send request for next line.
	if(retryTime > newLineTime + 10000){
	Serial.write('n');
	newLineTime = retryTime + 10000;
	}

	while(statusmsg == 'r'){
	delay(150); // 150ms delay
	read_data(); // Read data from Serial
	loopTime = millis(); // Start time indicating that data is being read.

	//Wait 10 seconds for buffer to read and store incoming data atleast 6 times.
	if(loopTime - startTime > 10000){
	strcat(msg, "..."); // Concatenate "..." at the end of data.
	clear_row(0); // Clear row 0
	clear_row(1); // Clear row 1
	display_msg(); // Display data
	clear_buffer(); // Clear the msg buffer for next line.

	delay(300); // Wait 300ms

	clear_row(0); // Clear row 0
	lcd.setCursor(0,0); // Set the cursor at position 3 to print Title Message
	lcd.print(" Reading "); // Set wait message
	lcd.setCursor(0,1); // Set cursor to lower row
	lcd.print("Deliciousness..."); // Set Copyright Info
	lcd.setCursor(0,0); // Set cursor at top left corner of LCD screen

	Serial.write('n'); // Request next line
	newLineTime = millis(); // Record the time next line was requested
	statusmsg = 'd'; // Set status message as done reading.
	}
	}
	}

	// Clears the buffer
	void clear_buffer()
	{
	memset(msg, 0, 400);
	bytei = 0;
	}

	// Clears the row provided in the arguement
	void clear_row(int r)
	{
	lcd.setCursor(0,r);
	lcd.print(" ");
	lcd.setCursor(0,r);
	}

	//Reads data from the Serial port.
	void read_data(){
	if(Serial.available()>0){ // If the Serial port is available start reading
	while(Serial.available()>0){
	msg[bytei] = Serial.read(); // Store the data in the msg buffer
	bytei++;
	}

	Serial.println(msg); // Print to Serial to debug
	}
	}

	//Display the message onto the LCD screen
	void display_msg(){
	// Display the displayname of the page data is pulled from and center it onto screen.
	for(bytei = 0; bytei <15; bytei++){
	if(msg[bytei] == ':'){
	int diff = 15 - bytei;
	lcd.setCursor(diff/2 , 1);
	lcd.write('#');
	for(int i = 0; i<bytei; i++){
	lcd.write(msg[i]);
	}
	lcd.setCursor(0,0);
	break;
	}
	}

	// Display first 16 characters of msg.
	for(bytei = 0; bytei < 15; bytei++){
	if(msg[bytei] != '\0'){
	lcd.write(msg[bytei]);
	}else{
	//lcd.print(' ');
	}
	}
	bytei=0;

	// If there are characters which are off screen, shift the characters and scroll
	while (msg[bytei+15] != '\0') {
	lcd.setCursor(0,0);
	for (col = 0; col < 16; col++) {
	lcd.write(msg[bytei + col]);
	}
	bytei++;
	delay(350);
	}
	}

view raw GoogleReader hosted with ❤ by GitHub

Upon power up the Arduino displays a welcome message and then waits for data from the serial port. If 10 seconds pass and no data has arrived it sends a request for the next line. Ill explain why it was done this way later on.

Once the Arduino receives a ready bit from the python script it gets ready to receive data from the python script. It receives data in chunks of 64 bits and tracks the time from when it first started receiving data. It waits for 10 seconds till all data has arrives and then displays it on the screen. If the data is longer than the screen width then it scrolls it across the screen.

Once its done scrolling it requests for the next line from the python file.

During testing i found that sometimes the python script would keep waiting for the request to transmit the next line. To prevent the script from waiting forever, the arduino sends another request if it has not received anything for more than 10 seconds.

And thats pretty much it.

Lets see how it works!

Hope you guys enjoy!