I understand your point. However I run several lists and prefer to manually approve serious mails instead of spamming every member by keeping the gates open. It is a bit more work but keeps frustration away from list members. Of course this is just an opinion.
Instead of simply rejecting them, moderate all the non-subscribed messages. The best of two worlds.
Manual approval from nonmembers, letting through anything vaguely on-topic written by a human, is not onerous and wouldn't break expectations.
Letting one annoying message slip through for every 100 is quite acceptable.