When I was younger my grandma would only use pencil to write down my address in her book. She said I moved around too much to use pen. When you move that often, you tend to have amazing memories of your friends from those chapters, but you don’t really get the experience of growing with them.
Mandy is my oldest friend, going on 11 years. I love and cherish our friendship, and I’m so thankful that we have grown up, and remained friends. Mandy has an 8 year old daughter names Abbey. Abbey is autistic and completely non-verbal. I have seen tears of frustration in Abbey’s eyes, when she can’t communicate when she needs. It breaks my heart to not be able to help her, and I can’t imagine what my friend must go through daily. My desire to help and the amazing aura of the D365 Community is what got my wheels spinning on Abbey’s App.
First, I saw Phil Toppness do a demo at a D365 Saturday back in June 2019, using the Face API as a mood detector funny app. Then, in September I participated in the TDGI Global Hackathon. Some of the prompts revolved around gamifying learning, and using AI to help children with special needs. This reminded me of my special friend Abbey, and how I have seen her struggles when she gets overwhelmed in social settings, but doesn’t know how to process or communicate to mom how she is feeling. I also know that part of what helps Abbey calm down in these situations is some valued screen time, and her LOVE for selfies. There was only 5 hours in the hackathon, and I knew this app was too much to tackle in that time frame. However, my mind kept on ticking, and I had to #ToDoTheThing.
The idea was born! Create an app that has “Learn, Practice, and Play” stages. The learn stage shows a couple of videos/photos of social situations, and the appropriate response for those emotions. I did some research and found 6 basic emotions to focus on. Some of them are easy like happy and sad, some are more difficult, like surprise or scared. By emphasizing on different situations, and the facial expressions, the child has a frame of reference for how they should respond.
The next phase is hands on practice!! The child will be asked to take a “selfie” of themselves showing the emotion given, (i.e. “Happy”) Using Face API, feedback will be given on the level of “happiness” the child showed in their picture. This goal, or objective will be for the child to reach a desired level of happiness before they can move on to the next phase.
The Play phase combines the first two scenarios. The child is shown a video clip, but instead of being told what emotion is appropriate, the child will take a selfie to show the correct facial expression to show the emotions appropriate for the situation.
I quickly realized this app contains some concepts and components that would take me a REALLY long time to figure out. I was asking the community (D365, PowerPlatform, etc) a lot of questions, and had a lot of positive feedback and excitement over the project. I knew that if I could turn this project over to the community per say, and just try to provide an outline, and clear objective, that the end results would be far greater than anything I could dream of on my own.
So a few tweets, later, and lots of mistakes, a plan was set in place to meet on December 7, 2019 from 10:00am-3:00pm. Boy oh Boy… did I get in over my head, but that’s been half the fun! I plan to blog my series of mistakes and failures of what is now my little #CommunityHack4Good event.
There are MANY challenges and limits, but I’m okay with all of that. My goal is for the community to be able to collectively put together a workable prototype that can essentially go through a CRP 1 type phase. I hope that it sparks all kinds of thoughts in the imaginations of those far smarter and savvy than myself, with a result of having a true impact in helping children with autism learn how to better understand how to express their feelings.
I’m not an expert in any of this, I’m just a person with an idea, excited to be a part of an amazing community, that is working together for the betterment of others. Thanks for taking the time to read about the start of this wild ride, I hope you will follow our journey.
