Loose rudder
It is hard to say from your description if the rudder is loose or not-my good friend "valentor" gives very good advice on how to check it.
What you are seeing is likely this:
1). Responsive to pumping in current: Your boat has a pretty small rudder, and there are times when it can't overcome the forces acting on the boat-these could be current, heel, weather helm, etc. and a vigorous pump may help at these times. This is not a complete explanation but is nonetheless true.
2). Vibration at 4+ knots: Far and away the most likely culprit is an unfair rudder-meaning that the sides are no longer the smooth, shaped surface they were built with, or the trailing edge may have dings or dents (for that matter it could be the leading or bottom edge also). Any of these factors can and often do produce a vibration and sometimes a humming noise. This is more likely the cause of the vibration that a loose rudder-since we see it in brand new rudders all the time(until they have been faired). Next chance you get, make sure both sides are smooth and symetrical, and that the front, back and bottom are straight and free of dents or dings. I'll bet you a buck or 2 you will find one or all of these things with your rudder. Fix them and it should go away. The speed at which the vibration occurs can vary from 1kt.-hull speed-all depends on what is wrong-but can't be predicted.
Also possible, but much less likely for you, is an unfair portion of the keel-again, could be on any edge or side of the keel, but sometimes this will cause a vibration and/or humming, felt through the steering system. If your rudder is perfect, check the keel-I am sure you will find something amiss-and it doesn't take much to cause this.
Good luck!
S