Tag Archive: C#

Object Recognition SDK

Object recognition on mobile phones is typically done by uploading a picture to a server for recognition while the user waits. This project took recognition algorithms and the image database and put them on device. Applications using the SDK will include games, educational tools and real world browsers.

The performance is impressive. With 1000 images in a database, the typical Android device can make a recognition in about 0.25 seconds. Images can be recognized at a sharp angle or even with the device held upside down. Partially hidden objects are also be recognized. To generate their own database, users can also add their own images from the camera phone.

The starting point for the SDK was some experimental C code for database creation and recognition. It needed to be refactored and made into a native Android library using the Android Native Development Kit (NDK). That library is called by both a PC-based database creation program and by a reference application that SDK users will use as jumping-off point for their own projects.

The SDK is available for educational use and may be released later for commercial development.

Perfect Drink: Become a bartending superhero

drinThis was one of the funnest projects ever to test! Working for Perfect Company, I developed the Android version of an application that shows you how to mix hundreds of drinks. It’s currently sold through Brookstone stores and online. It’s been featured on the Today Show and in Time Magazine.

What’s unique is that the user needs to purchase the hardware — a kitchen scale that connects to your device via an audio cable. The tangible product concept has allowed the product to sell at a higher price than software titles for mobile devices.

The experience goes like this. Browse the recipe book by category or by what you have in your cabinet. Each drink recipe and ingredient includes interesting history and notes. Select a drink to build and you are prompted at each step. As you pour, the screen shows the current level. If you overpour, the rest of the recipe can be scaled and adjusted.

The UI looks great both on phones and tablets and the app makes use of a core module that is common to both the IOS and Android platforms. This core includes both database and scale interface functionality. The Android app can access it via a SWIG interface. The app has been globalized and to date has been translated into 7 languages.

Download for Android.

MyTaxBack

One Economy has provided free tax filing service for a number of years. The benefit ensure people entitled to the Earned Income Tax Credit to receive it. Many don’t file and therefore miss the benefit one of the most effective poverty fighting policies in the country. The average refund is $1500.

MyTaxBack was a devised as a way to alert people to how much money they could put back in their pocket by filing. The app was developed for SMS, desktop browsers, Facebook, iPhone and Android. We used JQuery, custom CSS, PHP and PhoneGap.

You can try it here.

As part of the MyTaxBack program I wrote a reusable SMS response tool for text messaging campaigns. We also created a database for tracking users of the various OE programs and media, anonymously, of course, to be able to reconnect at opportune times. For example, if someone files for a tax refund, we would send a text reminder, at about the time she would receive the money. The message would remind her check out options for setting up a bank acount or paying down credit card debt. The vision is to create sort of a CRM for coaching people to help themselves.

Ensuring Rest for Drivers

Many people don’t know that Qualcomm got its start in communications for long haul trucking. On my first project as a contractor there, we specified and wrote the  Qualcomm Hours of Service, a driver-tracking package used on most long haul trucks in the United States.

As you go down the freeway, notice the white dome on top of many truck cabs. This sends signals to a satellite about the truck’s position. The driver can also send and receive short messages from a device in the cab.

The drivers are required by law to rest a certain amount and the rules and monitoring are rather complex. Using C# and ASP.NET, we created the database, services and web site that tracks thousands of drivers and trucks in North America to be sure they stay in compliance.

Short Voice Service

Qualcomm has been very invested in the wide adoption of devices that use data as well as voice. As well, serving the next billion cell phone users was at the forefront of strategists minds.

I got on a team creating prototype applications for emerging markets. In this capacity, I wrote Short Voice Service, a BREW reference application for peer-to-peer voice messaging, a variation of which is now in use in India.

The peer-to-peer aspect is interesting. Phones need to make a data connection to send and receive data. They don’t keep an open channel and they get a new IP address each time a connection is made. To bypass the need for a server, the sender records the message, opens a data call, gets an IP address and sends it in a text message to the receiver saying “there’s a voice message at 10.23.33.125”. The receiver then opens a data call to the waiting sender and pulls the message across. The transfer can also occur with a server intermediary.

Being able to send a voice message from phone to phone has some kind of interesting attributes not found in other channels of communication.

  • It’s like texting, without the typing.
  • It’s like voicemail, without listening to the recorded message telling you what to do.
  • It’s a message without the conversation.
  • It conveys more emotion than email.

A funny ad came out that illustrates the importance of properly creating that emotion:

youtube=http://www.youtube.com/watch?v=EsmdA6TCC1g

Image-based Search

Qualcomm holds an annual innovation contest to harvest new business ideas from its offices and labs around the world.

My team came up with a winning business plan and was able to start a new research initiative in image-recognition-based search using mobile phones. I pitched in by writing part of the business plan, the BREW and Windows Mobile client demo applications and the project web site. The demo could identify thousands of beer, wine and DVD labels.

While the idea did not fit well in the company’s business model, developing the technology to make Qualcomm chipsets excel when used for image recognition was identified as an important objective.

The concept has actually hit the market via Google with their Goggles product seen below as predicted in our competitive analysis. Taking a picture of labels,text, bar codes, and one day even faces leads to the search that the app thinks you are likely to want.

Combined with GPS, previous behavior, time of day and other inputs, the search intelligence of mobile apps will continue to grow.

WindowsMobileTraining

ViaTraining, now part of General Physics, creates enterprise learning management systems. Their original system was based in LAMP and they had a contract with Microsoft who, not surprisingly, preferred that ASP.NET and SQL Server be used.

I helped with the port to .NET technology and globalized the interface. Windows Mobile Training was in use for about seven years and typically had about 10k users in 13 languages at retailers like Best Buy.  The system managed their training progress and dispensed incentive rewards.

Motor Sports Scoring Site

This project was for a small startup that was creating a site for kart racers and their fans to see race results. There is a very enthusiastic culture of kids and their parents who go to weekend races. It turns out that the tracks all use an automated timing system that counts laps and tracks times using transponders attached to the karts. I deciphered the data format of the timing system, byte-for-byte and added various enhancements to the web site using ASP.NET and C#.