Work / Product
LisNav
Real-time AR audio descriptions that guide visually impaired people through the city. Co-funded by EU Urban Innovative Actions.
- AR
- AI
- Accessibility
LisNav sits at the intersection of augmented reality and artificial intelligence. It turns the street into real-time audio descriptions, so visually impaired people can move through known and unknown places with more independence and confidence.
- Platform
- iOS, on the App Store
- Funding
- EU Urban Innovative Actions, EIT
- Award
- Jóvenes máshumano 2024
Under the hood
Explained from the system’s own source code.
Sense
Audio mode uses GPS and motion. AR live mode adds the camera through ARKit, with an on-device Core ML model (YOLOv3) recognising objects in real time.
Understand
Nearby places and hazards come from open data and community reports, scored by distance and relevance. Camera assistants describe a scene, read text aloud and scan codes.
Speak
A narration layer turns every moment, from a place ahead to a route step or a hazard, into one short spoken line, with adjustable verbosity and voice.
Guide
Walking routes with spoken turn-by-turn guidance and route previews, through the phone speaker or any headphones.
The work in use
Explore: audio-first awareness 
Navigate: spoken guidance 
Safety: hazards nearby 
Profile: voice and verbosity
Accessible bus stops 
Community content 
Aligned with UN SDGs 3, 8, 10 and 11 Walking Lisbon with live audio description
What it does
Audio-described streets
AR geolocates points of interest and AI describes them in stereo audio, through the phone speaker or any headphones.
Accessible bus stops
Clear information on bus stop names and connections makes public transport easier to use without sight.
Open data, enriched
Publicly available datasets feed the descriptions, alongside points of interest added by the community.
Collective safer journeys
Users report hazards, emergencies, road closures and points of interest, and everyone gets the updates in real time.
- Part of VoxPop, fostering the digital transition of mobility in Portugal.
- Co-financed by Câmara Municipal de Lisboa, Câmara Municipal de Aveiro and the European Regional Development Fund through the Urban Innovative Actions Initiative.
- Supported by the European Institute of Innovation & Technology (EIT) through the Better Mobility Accelerator.
- Jóvenes máshumano Award 2024, Fundación máshumano & Bankinter.