We find candidate boards by their edges and save them as rectangles. These rectangles are evaluated to determine which are really boards. The amount of content on a board determines when last it was used. All the boards are then captured in an enclosing frame.
We detect movement by comparing the differences between frames and storing them in rectangles. The lecturer is then found in one of these rectangles based on the time spent on screen.
We use lecturer positions to decide whether (and how) to pan the virtual camera such that both lecturer and the board being referred to are kept in-frame.