Why is knowledge base search so hard to get right?
Because the knowledge is not in the knowledge base. Every company that has run this exercise finds the same split: the documented answer exists for perhaps a fifth of the questions people actually ask, and the rest live in chat threads, ticket resolutions, email replies and comments on a doc nobody reopened.
Native search inside a wiki is generally fine at its job. The failure is one of scope, not relevance. Confluence search does not know your Slack exists, and a help desk search cannot read the Drive folder where the runbook was actually written. Fixing this means one index across all of them, not better queries inside one of them.