@Hanoi IG since I'm apparently the only person on the other side of this I guess I'll chime in again.
You had a plan. You followed through on the plan. You paid a bunch of origination fees, closing costs, appraising fees, whatever.
Those are all sunk costs and I'm not trying to tell you to "take them into account" sunk-cost-fallacy-wise. I'm telling you to take into account the things you were thinking that led you to make the decision in the first place. If something has changed since you made that decision, or if you have learned more things and consider yourself better-informed and better-educated about the risks and rewards of investing in the market while carrying a mortgage, good, reassess your situation and potentially arrive at a different conclusion.
On the other hand, if you made the decision impetuously, and are now making another decision impetuously, then your problem is making impetuous decisions, not carrying a mortgage while investing in the stock market! Fix the real problem, which is not your mortgage/stock allocation.
And finally, you need to consider the possibility that your younger self was right. Maybe it was that younger self that thought through everything and decided that carrying the mortgage while investing in the market was worthwhile, and he thought through all the risks and rewards and arrived at a sensible conclusion, and would be furious that you, older self, are second-guessing his careful homework.
I'm not saying "don't pay it off," I'm saying don't treat every financial decision as an in-the-moment, do-I-go-left-do-I-go-right, split-second decision. Make decisions that you have enough confidence in that you'll stick with them for 5-15 years, and reevaluate them only when you have significant, new information.