Pointer events are difficult in a network context, because it is very difficult to make the delay not noticable to the user. Using TCP_NODELAY to disable Nagel's Algorithm, and UDP do not seem to make a significant difference.
On the other hand, the mouse is a relative newcomer in popular computing. Computers were used for years and years very productivly without a mouse. I am thinking of taking out, or at least de-emphasizing the mouse, and just pressing on in other areas. I can always re-visit the issue later when and if I have more resources than I do now. This way I will not get bogged down in one technical area.