With the development of online lectures, the teaching quality comes in front of us. Specifically, due to the position setting of the camera, it is inevitable that teachers may occlude a part of the contents on the blackboard during giving a lesson.
Hence, we propose an approach to reappear the occluded blackboard contents. As shown in the above figure, we firstly extract the human mask from current frame, and then search an appropriate past frame which is able to provide the occluded contents.
On account of some complicated situations, e.g., teachers hold something in their hands while teaching, we take the segmentation as saliency detection instead of vanilla semantic segmentation with a human label.
With great efforts on model improvement and multithreading deployment, we provide a blackboard reappearance solution that can inference reliable human mask from a HD frame in real-time with an NVIDIA GTX 1080Ti. In addition, this project has been exhibited at the 77th China Educational Equipment Exhibition.
The following video shows that our model has the ability to produce trustworthy saliency area and the proposed solution is successful in reappearancing the blackboard contents and further improves the quality of online courses.
* In particular, thanks to Gang Dai for his help on this project!
