When to Be Proactive and When to Stand Down

Summary: The short answer to your direct question is that you certainly could tell people to leave you alone. If you aren’t angry right now, it might be sending the wrong message. I assume from the way you describe them coming in often to share ideas that you have, heretofore, operated with a bit of an open-door policy and an open attitude.