If I may step in, the current draw and power loss from a regulator not plugged into anything is going to be miniscule. With something like this, we're talking milliwatts at most, and even that's just so the tiny chip that does the job keeps humming over.
I wouldn't worry about putting a switch on it out of concern for the regulator itself draining the battery. If you want to avoid accidentally draining the battery because you forgot your phone/tablet/gps was plugged in then THAT is a good reason to put a switch on it.