We have developed a basic Arduino library that allows an ecosystem of IoT devices (based on ESP32) to send their data (audio, images, measurements) to our backend.

Every communication (both REST calls and Web Sockets) is protected by JWT and profiled against a Postgres database.

Our backend is developed in Nest.js and accessible as a Web API.
The backend is hosted on Azure and is highly scalable and extensible with the addition of new AI services.

We have developed a Typescript framework that allows any “JavaScript-enabled” device (smartphone, tablet) to act as a “client”.

The framework is “ platform agnostic ” and uses a list of Managers (basically interfaces) to interact with the backend and manage the device (e.g.: get audio from a microphone, display the recognized voice, play a TTS voice, etc).
Using this strategy we are able to quickly develop a PWA (Progressive Web Application) for a specific device (e.g. a tablet) simply by implementing the bindings to the hardware (thus “linking” the Manager implementation for that hardware).