Teachable Snake is a classic snake game controlled by webcam image, powered by Tensorflow.js and Teachable Machine by Google, using pre-trained neural network models.
Google Creative Lab came to our class to introduce the latest version of their Teachable Machine, which provides ability to store trained neural network model in the cloud, and using it in other projects outside the Teachable Machine website. Impressed by the project, I decided to use it for my final project in the class.
The project is inspired by the Webcam Pacman project. The idea here is to create a classic snake game powered by machine learning neural network controller instead of traditional physical ones.
The framework of this project is Create React App. To manage the complicated state of the snake and the controllers gracefully, React Hooks has been an incredible helpful tool. And to pave a way for future possibilities, including multiple players, the code is structured as modulized as possible.
At first, the website was running extremely slow and consuming a lot of CPU power. I thought the issue was coming from memory management of
Tensorflow model. But turned out it was because I was calling the functions inside of React hook
useEffect API, instead of traditional
componentDidMount life cycles, the mobilenet
loadModel() function was accidentally being called repeatedly along with other looping functions. After I refactored the structure to set
loadModel() function to be called only once after component mounted, the website seems to be working more smoothly now.
Using the trained model provided by Teachable Machine is pretty straight forward. Just load the model and webcam in the first place, and make prediction in every
To make improvements on this project, other than enhancement of the user flow and user interface, the ambitious goal is to setup a
websocket for multiple players to experience the game at the same time!