Posted by Vignesh Prasad (Student, GSoC 2015)
This blog post has a lot of meaning to me for 2 reasons. It marks a prominent leap in my first noteworthy contribution towards open source software (OSS) and because this is my first blog post. In this post, I would like to walk you through the app that I have been working on over the past month or so. It is an educational app that I am developing with BuildmLeam as a part of Google Summer of Code 2015.
With their increasing popularity and low cost, smart devices like smart phones, tablets or in the recent years, smart watches and smart TV’s, have made their way into almost every household that we see. Kids these days are also quickly getting oriented towards technology and the time has come for educational methods to do the same. We have seen the effect of technology on education as well with the arrival of smart classrooms. It’s about time that this effect reaches a larger scale. We hope to do that with this app. The main aim, as the title suggests, is for users to practice handwriting. I see it as an app that is aimed mainly at children. It is something that can be widely used in primary schools and kindergarten for doing so.
This journey of mine started around the end of April when the GSoC ‘15 results were out. A friend of mine who was in my room at the time said that my name wasn’t there in the list. I was a little disheartened and went back to sleep. As soon as I hit the bed, he woke me up saying that my name had appeared on the list! In a half asleep state of mind, it still hadn’t hit me that I got into GSoC ‘15. The next morning, I woke up happily and reopened the GSoC website hoping that the previous night wasn’t a dream. An email from my mentoring organization (org in short) confirmed that I did get into GSoC ‘15.
It’s hard to believe that 2 months have passed since that day. I’ve come a long way and learnt new things since then. My GSoC journey started in the community bonding period when my mentors and I were finalizing the UI and flow of the app. I had never worked on UI much so it was a bit difficult to come up with a design that was appealing to children so that they wouldn’t get bored and at the same time, keeping it simple enough for them to use without much navigation. UI wasn’t one of my stronger skills but I feel that in these 2 months, it has improved.
Once the UI was finalized, it wasn’t such a difficult task as a friend and I had worked on a similar app, TrazeMaze, a long time ago. This allowed me to implement the basic practice session pretty quickly and gave me time to focus on the new features that were planned. Another thing I learnt is the implementation of animations in the android framework, something that I’ve been meaning to do for a while. This was done since the app, as mentioned before, needed to be appealing to kids and what more could capture a child’s vision than sliding/rotating buttons, fading texts etc.
Now coming the the app. It speaks the string to be practiced and displays it to the user who has to trace within the boundary of the string. If the trace is going wrong, the app gives an audio-visual and haptic feedback to the user signalling that the trace has gone outside the boundary of the string. Once a user is done, each trace is given a score based on the number of correct and wrong touches. This introduces an aspect of gamification of the learning process which helps to enhance the learning experience. Users also have the option of saving their traces to the gallery and view it later.
The tracing framework was implemented using a custom view named DrawingView with the help of this tutorial. It was done by drawing the text to the DrawingView’s canvas and checking if the colour of the pixel touched by the user is the same as the background or the text. The device vibrates if the user traces outside the boundary and displays a Toast if the user stays outside the boundary for too long.
Coming to the extra features, I have implemented timed tracing and freehand tracing of characters. This is when the the task got challenging and started to tickle my brain. When I had proposed the idea of borderless tracing, I wasn’t completely sure of how I would implement it or how it would come out. It took me a while to think properly on how to approach the problem of scoring a borderless trace.
After a few days of brain storming, I used a combination of techniques that were taught to us in a computer vision course. *Technical details coming up* Those who would have done a course on it would probably know what a scale pyramid is and how a HOG filter is applied at multiple locations in an image to obtain the maximum response. What I finally ended up doing was something similar, finding the best score that could be obtained over multiple scales of the user’s trace, at multiple positions near the center of the view where the character is placed. It still needs to be a little fine tuned, but in my view, the results are pretty decent.
Once this was done, I quickly started implementing the time trial feature and got it working quickly. I thought that implementing the time trial feature would be easy but it turned out that I forgot to consider memory constraints in my approach of saving all the bitmaps of the traces. When I got this error after storing the traces of about half of the characters on a 4 inch screen, I could only imagine what it would do on larger 8-10 inch tablet devices. After few more days of trying various tricks to prevent the error from occurring, I took a new look at the problem. I decided to save the user touches and then redraw them on a new view while displaying the results. There was an initial pothole on this road when I was trying to send the touch events to the view but that was taken care of.
Finally, the first version of the app was ready. I took a big sigh of relief and gave myself a pat on the back on completing the app. I had sent the results to my mentors and was in a relatively happier state as compared to the last few days. Everything was going fine. Since I am from India, the land of Bollywood, my journey involved a victory song that was playing in my mind for my success. Who would have known that this journey would take a plot twist as typical as in every other Bollywood movie but without the dramatic expressions. It turns out that the app was working fine on my devices but kept crashing on my mentors devices.
*More technical details coming up* Following the pats on my back were scratches on my head as to what could have gone wrong. After a few emails and crash reports, I found out that my approach itself was flawed. I was creating a TextView with the string and taking a screenshot of the TextView and setting that image to the Canvas of the DrawingView. There was a way to set the text itself to the DrawingView Canvas but it initially hadn’t worked out well for me. After a little experimenting, I managed to get it to work as well as my initial approach did.
Once all that was finished, a few bug fixes here, a few animations there, and voila, PracticeHandwriting v1.0 is ready for testing. For further steps, once the bugs are fixed, I plan to add tutorials for first time users and add support for Hindi and Arabic as well. The unique aspect about the tracing framework is that only the string to be traced has to be passed to it. This facilitates the easy tracing of characters and words of any language without the need to change any part of the workflow of the app.
There is still a long way to go before the app is ready for a large scale release. I hope that the app sounds interesting enough to all of you reading this post. In case anyone wishes to take a look at the source code, you can find it here. To quote Arnold Schwarzenegger from the Terminator movie, “I’ll be Back”. Check again within a few weeks for more updates regarding the release and other details of the app.
Originally published at http://vigneshprasad141.tumblr.com/post/122976126295/practicehandwiting-changing-the-way-kids-learn