Melsonic is a UK-based music-EdTech startup helping beginner and intermediate guitarists improve through real-time AI performance analysis. We integrated Melsonic's existing AI engine into a fully-built Next.js web platform, Web Audio API capture, pixi.js note rendering on a tightly-synced canvas, AI scoring against the original track (notes correct, missed, played wrong), and a gamification loop that keeps players coming back. Stack: Next.js + pixi.js on the front, Nest.js + PostgreSQL on AWS at the back.
Sync real-time audio with note rendering, and turn raw performance data into feedback a player actually trusts.
Melsonic had a working AI engine. They needed a product around it: a web platform that captures a player's recording, syncs note rendering on a canvas to the live audio, runs the AI comparison against the original track, and renders feedback (notes correct, missed, played wrong) in a way a learner can act on.
- Syncing real-time audio capture with note rendering on a pixi.js canvas
- Smooth user experience during recording, latency-sensitive across browsers and devices
- Transforming raw audio analysis into structured, motivating feedback
- Hitting a five-month MVP target with a full product surface (auth, learning, recording, feedback, profile, gamification)
Next.js front, Nest.js API, pixi.js canvas, wrapped around Melsonic's AI engine.
We built the full web platform on Next.js, with a pixi.js canvas for note rendering tightly synchronised to the audio capture pipeline. A Nest.js API on Postgres + AWS sits in front of Melsonic's AI engine and feeds the scoring loop. Seven core modules shipped, sign-up & login, landing, learning, feedback, recording, profile, and gamification, into a five-month MVP that's now an ongoing engagement.