Brownie - Automate Your Home with AI

Do you need to know if a door is open or closed, if someone is coming, or if a trash can is full or not in your house with an AI camera? It's cool!!

When you start to use machine learning, especially using deep learning, you might find that you must have a lot of images to work to improve generalization performance.

But wait, is such generalization performance needed at home? Why do you have to take many pictures to create a model? It's so hard...sigh

So, I made an easy-to-use AI camera app, that needs only one image to work well at home.

How do you do that?

1. Use a deep learning model as a feature extractor. In other words, automatically find nice features in the image with a model that was trained with lots of images.

2. Compare the features' similarity with a classical and simple machine learning algorithm such as k-Nearest Neighbor, and find the nearest feature.

That's all. It surprisingly works at home.

Here is an example of it.

Setup for M5StickV device

1. Download MakerFaireTaipei2019.zip at https://github.com/ksasao/brownie/releases/tag/v2.0 .

2. Unzip and copy all files under the M5StickV directory into your microSD card.

3. Insert the card into your M5StickV device.

4. Connect your USB C cable with your PC and M5StickV device.

If you need to fix your device firmly to a tripod, you can print the spacer below with your 3D printer.

Setup for your PC

1. Copy the Windows directory in the zip file above to the desktop of your PC.

2. Double-click a BrownieMonitor.exe to run the app.

3. Input the COM port name your M5StickV is connected to.

4. Push the "Connect" button.

How to use

1. To associate an object with the name, prepare a QR code that has the name of it. You can use the QR Code Generator https://www.the-qrcode-generator.com/. You can also use the QR.pdf included in the zip file above.

2. Present a QR code to the camera to learn a scene or item. When a red frame appears, you remove the QR code and show a scene or item you want it to learn. To reset the learned item, just show a *reset QR code to the camera.

3. Verify that the target is recognized correctly (Green Frame will appears) and check the name shown on the Brownie Monitor app.

4. Brownie Monitor will send a command when it receives the name from the camera. Any command will be accepted such as python. Use start to launch the default app associated with file suffix (e.g..mp3,.png,...).

Application examples

• Play a welcome sound when a door opens
• Turn off the light when you sleep in bed
• Switch what is displayed on the screen when someone comes behind you :-)

If you find a cool use case, please tell me on Twitter (@ksasao)!

Custom parts and enclosures

Tripod attachment for M5StickV