Our society often acts as if sight is the only sense that matters. We are more impressed when we are visually stimulated. We are surrounded by big visual stimuli and there is a clear prevalence of sight over the other senses.
The truth is that hearing is vital in our environment: the main communication channel us human beings have is the verbal one, which we need the hearing sense for; music is one of the finest human art expressions, which can not be fully appreciated without our hearing sense; and we do not usually take as much care of our hearing as we should.
Mimi is one of the companies that realized the importance that good hearing has on people’s lives. Their team is devoted to contributing to people’s hearing health, building products that adjust the sound to the hearing of the individual. People’s hearing depends on their socio-demographic characteristics. As we age our hearing gets worse, and beyond that exposure and genetics also affect hearing. Concretely this means that most people do not hear perfectly anymore.
Wouldn’t it be great to adapt the sounds you hear specifically to your personal earprint?
This is especially important for people who have hearing difficulties. As Mimi points out, people with hearing difficulties often compensate it by increasing the volume of what they want to hear. These people often suffer from subtle deficits like sounds blurring over time and across frequencies. As a consequence, they have a hard time separating individual sources in complex sound environments. Simply making sounds louder does not help these more delicate aspects of hearing impairment, and can even make things worse!
Moreover, Mimi strongly believes that hearing is a basic human need worth protecting while improving the way you hear sound contributes to a better quality of life. That is why everybody should take care of their hearing, not only those with difficulties. Unfortunately, too few are currently aware of their hearing capacity as well as the solutions to potential hearing deficits.
With that basis, Mimi has developed a fantastic algorithm to adapt the sounds to each particular user’s needs, according to their individual hearing. Their goal is to reach as many people as possible. One of the best ways to do this is through a device that is globally spread throughout the world: the smartphone. And here it is where TipTap comes to help: Mimi and TipTap have signed an agreement to integrate Mimi’s technology in the Android operating system.
This integration presents several challenges, such as the device fragmentation, the audio latency issues in Android, the variety of libraries to process audio or adapting to different sample rates and buffer sizes.
We started by digging into the not-so-extensive documentation about audio processing in Android, to learn about the different ways and best practices to handle audio. We performed tests using three different audio engines: Superpowered, Audiotrack and OpenSL. We picked a very extensive selection of devices with different form factors, resolutions, and operating system versions to compare results.
Although Superpowered and OpenSL are great alternatives for the audio processing, we finally picked Audiotrack as we saw that in our testings it performed fantastically across all the devices we tried and, on top of that, it offered very good quality with the latest Android versions.
Once the decision was made, we started our work to create an Android app capable of improving (“mimify”) audio according to some parameters uniquely configured for each user.
Some of the most remarkable features this music player has are:
- Personalization. Users are guided through the registration/login process so that they can configure their hearing profile. Users go through a set of configuration screens that will allow them to equalize different frequencies for each ear. Alternatively, users will be able to configure the settings by simply choosing their age and sex. (Soon they will be able to do a hearing test that will configure everything in an even easier way)
- Media library access(“My sounds”): all audio files in the system are listed to the user, sorted by Title, Artist, Album or Genre, ready to be mimified.
- Mimifier: the player main screen allows the users to mimify their music adaptively from 0% to 100% of mimifying, by manually tuning a slider. This will introduce fantastic improvements in the audio processing the file played.
The implementation of the app was done based on a new SDK we also created. This SDK, apart from being leveraged by the app, has been created to provide Mimi’s potential partners with a means to use the audio mimifying process within their very own third apps. The SDK will allow this partner to reproduce the audio of their apps according to the user’s earprint.
If you are interested in testing Mimi Music, we encourage you to download the application directly from the Google Play Store and give it a try. You may also want to check Mimi’s developers’ site to learn more about how to get the SDK and use it in your own apps.
One of the values we have in TipTap is to contribute to the creation of a better society with our products and work. And we absolutely agree with Mimi that improving your hearing impacts in your physical and mental wellness. This is why collaborating with Mimi is being a perfect match for TipTap.
And this is just the beginning! Everyone should be able to listen not only to their music in all its richness but to any sound coming out of their phones such as ringtones, videos or even phone calls and video calls in the best way according to the each of the user’s unique hearing profile. We will be happy to keep collaborating with Mimi to let the maximum devices and users possible to enjoy a fully new and enhanced hearing experience.
Article written by Juan Pérez-Bedmar, Pre-sales Manager at Barbara IoT.