Cursor loses focus

When you hit escape a couple of times to use block mode and then again to exit block mode you end up completely lost without ability to get the cursor back to exactly where it was, unless you reach for the mouse again.