When Should You Take Money Out of Your Retirement Account to Pay Off Your Mortgage? Retirement Planning Strategies and Retirement in Your 60’s

One of the questions that I receive all the time is when I sit down with a family and they’re about to retire, and they have maybe five years left, seven years left before their mortgage is paid off. And they ask me, Troy, should I take money out of my retirement account to pay off my mortgage, because I don’t want to have any debt? Is that a good thing to do?

Hi, I’m Troy Sharpe, CEO of Oak Harvest Financial Group, CERTIFIED FINANCIAL PLANNER™ Professional (CFP®) and host of the Retirement Income Show.
Alright, so let’s look at a real world example from a mortgage calculator that I ran before we did this video. We’re just going to assume a $300,000 mortgage at a 5.5% interest rate, which is a payment of about $1,703 per month, $20,000 per year, taxes of $500 per month or $6,000 per year, and home insurance of about $100 a month or $1,200 per year.

So, the total monthly payment on this house is $2300 per month, which is about $27,000 per year. Once we get through year 24 here, we have about $100,000 balance left on that mortgage. When we pay this $27,640 for the payment next year, because this number doesn’t change, it probably actually goes up because your property taxes probably increase over time, but when we have $100,000 balance left, we’re still paying $27,000. The thing that changes over time is the percentage of your dollars for that home mortgage payment that go toward principal versus interest.

So, as you own the home for longer and longer and longer, more and more of your dollars go towards the principal balance, which is building equity — it’s like a safe savings asset. Now, real estate can go down, so I shouldn’t say that it’s safe, but historically real estate, primary home ownership in this country has been a stable place where people can invest their dollars making extra principal payments, for a rainy day down the road or if you sell that house. But it’s not volatile like the stock market, or private equity or other types of investments.

So, we’re still going to pay this $27,000, even in year 24, with a $100,000 balance, the difference is the interest out of that $27,000 is only $5,149. If we look at this mortgage calculator, compared to the beginning, when we first got this mortgage. . . So, the amount of interest we initially paid was 13.75%. So, we’ve been paying more and more towards principal, as time goes on.

You go down here, this is about 10 years into the mortgage, we’re paying $13,000 in interest, about $7,000 towards principal. We get down here, so 17 years in, about $10,000 towards interest, $9,000 towards principal. So, that continues to. . . we continue to add more towards principle as time goes on.

Now, should you take money out of the IRA? If you take money out of the IRA to pay off that $100,000 mortgage, well, you have to go in and take out approximately $120,000, I’m assuming this is about a 20% tax bracket. Some of you may be in a higher tax bracket, some of you may be in a little bit less, but if you’re going to take $120,000 out, that’s going to bump you up, you’re probably at least going to be in a 20% tax bracket.

So, you have to pull more money out, $120,000, to pay off a $100,000 mortgage. You’re losing $20,000 in taxes off the top by pulling that money out of the retirement account.
Now, the interest that you’ll pay is about $5,149 if you do nothing. The opportunity cost, which is what would we have done with that money if we had not pulled it out of the retirement account, assuming we could earn 6% on our retirement money, we would forego about $7,200 in interest.

If you’re thinking about doing this right now, with the amount of stimulus the Federal Reserve has put into the economy, with what Congress is likely to do after the election, another $1.8, $1.9, possibly $2, $2.5 trillion in fiscal stimulus. There’s so much money out there, it’s pretty likely that over the next year or two the stock market is going to go up.

So, if you’re thinking about doing this now, this is a very, very bad time to be thinking about doing this because the opportunity cost could very well be 20%, 25%, 30% — maybe even more — over the next couple of years.

We’ve seen this story before coming out of the ’08, ’09 financial crisis: The Federal Reserve pumped a small amount of money relative to what they’re doing now into the economy and the stock market averaged 15% a year over the next eight years. So, but I do want you to understand this concept of opportunity cost: What is the cost that we will forego by the opportunities that we could take if we left that money inside our retirement account?

So. One, we’re gonna lose $20,000 to taxes off the top. Two, we’re gonna have an opportunity cost, which if we look at the interest paid, subtracted from the potential interest earned at 6%, that’s another $2,000 in cost. So taxes paid [are] $20,000. This is a losing proposition: We do not want to take money out of our retirement account. The simple answer to this question is never — you never take money out of your retirement account to pay off your mortgage.

Because as I’ve went through, as time goes on, what you’re doing with that payment, you’re still gonna have property taxes and insurance, so that’s not going away. But a large portion of that payment is actually going towards the equity in your home, which is increasing your net worth. Doing something like this: paying taxes, losing money from interest that you could have earned otherwise, that is decreasing our net worth.
So this is a bad decision all the way around.

Now, if you have money outside of your retirement account, and you don’t have to pay taxes, let’s say it’s bank savings, now it’s a little bit different. I’m in the boat where home debt, your mortgage, this is what I would consider a good debt. It’s not credit card debt, it’s not an auto loan, it’s not money you’ve loaned out to friends or family members that you may never see again. This is debt that if we really had to pay it off because we got into a bind, we have the assets, we have the money, we have the wherewithal to go ahead and pay that off.

So, this type of debt isn’t going to diminish our standard of living. It’s not going to put us in the poorhouse, it’s not a threat to our financial security. But I also tell people, Look, if you just sleep better at night, and you feel more comfortable knowing that you don’t have a home mortgage, [then] sure, go ahead and pay it off — as long as you have money outside of your retirement account that you do not have to pay taxes on whenever you withdraw it.

Now, we want to make sure, of course, for some of you watching this video, taking $100,000, $70,000, $200,000 — whatever your mortgage balance is — taking that out of your savings to pay off maybe a low-interest rate mortgage, maybe you’re paying 3%, 3.5%, [or] you’ve refinanced over the past few years. Paying that off could subtract from, again, the opportunity costs from your potential to generate retirement income, because that opportunity cost is a compounding cost over the years.

Meaning, if we would have left this money in the account, let’s say for another 10 years, and we average 9% a year and our mortgage is only 3% a year, [then] by paying that mortgage off, we’ve forgone a lot of potential growth inside of our accounts — which then turns into income and financial security to help pay for medical care costs, long term care costs, just retirement income, travel, [or] whatever it is that you want to spend in retirement.

So, understanding good debt versus bad debt, opportunity cost, the (what we call) arbitrage, which is the difference between money invested and the interest rate we earn, versus paying off that mortgage balance at a lower interest rate. The difference between those two rates (what you can earn and what you’re paying), that’s positive arbitrage. So, that has a compounding value over time as well.
Long story short here: Do not take money out of your retirement account to pay off your mortgage. If you have money outside of your retirement account and it makes you feel better and you can sleep better at night, [then] sure, go ahead and pay it off. But do understand the financial impact that taking a large sum of money out of your savings can have on your future account balances, your future security and your future income. But [from] the retirement account: no. There’s one purpose for that retirement account and one purpose only: It s to provide you income in retirement. That’s it.

If you liked this video, make sure to share it with a friend or family member. Hit that subscribe button. Hit that thumbs up button. Hit that little bell to be notified when we upload new content, and we will see you in the next video to keep you more connected to your money.