I Have $1.5 Mil, How Much Should I Spend, When Should I Take Social Security and How Should I Invest

Troy Sharpe: Three decisions everyone has to make when they retire is, when do I take my Social Security, how much am I going to spend now that I no longer have paychecks coming in, and how should I invest my money? A lot of people, once they retire and they start withdrawing money from their nest egg, they want to get too conservative, and some people stay too aggressive. In this video, we’re going to look at how those three decisions are all connected and how they impact, not just your lifestyle today, but how they impact your security over time.

Hi, I’m Troy Sharpe, CEO of Oak Harvest Financial Group, certified financial planner professional host of the Retirement Income show, which you can hear right here on YouTube, and also a certified tax specialist. What motivated me to do this video or gave me the idea was I was sitting with a client recently, and he said, “Troy, you know what? I really don’t think I should defer my Social Security because I want to leave more money in my investment account today. He was comfortable with the market so he said, “I’m going to keep my money invested pretty aggressively, and ultimately, at the end of the day, I should have a lot more money because I’m withdrawing less now in taking Social Security.”

A lot of times, and I’ve had this conversation hundreds, if not thousands of times, over the years with clients, when to take Social Security, how it impacts the plan, where we’re going to get income from otherwise. What a lot of times people don’t understand is, if you take Social Security now, yes, you withdraw less from your portfolio, but because your benefits are so much smaller here, about a 25% reduction at 62 from your full retirement age benefit and a significant reduction if you were to defer to age 70, what most people don’t realize is that you take less money out of your portfolio now, but in your 70s and 80s, because that Social Security benefit is so small, you end up having to make much larger withdrawals and you become a little bit more vulnerable to what we call the sequence of returns risk later in life. That is if the market is down and we’re making large withdrawals that can really deteriorate your portfolio’s value, and the last time you want that to happen is when you’re 75 or 78 or 82. We’re trying to grow those balances. We have enough money for medical expenses, assisted living, long-term care, whatever we might need, or maybe it’s just inheritance or for charitable giving.

I’m not going to tell you what, what is right or wrong to do for your personal circumstances because your situation is unique. You’re different from everyone else watching this video but what I do want to convey in this video is how those three decisions when you take Social Security, how much you spend, and how the portfolio is invested, how they’re all connected, and then how the sequence of return can impact the portfolio at different times throughout retirement. That’s a mouthful, but hopefully, you follow me here, and this will make more sense as we go throughout the video.

Hey guys, brief interruption. Two important things right here. If you like the video, make sure to comment down below, like the video, and of course, subscribe to the channel so we can keep you more connected to your money. Number two, we are having our third annual investor summit upcoming February 23rd, 6:00 PM. It’s a Wednesday. It’s right around the corner. There’s a link in the description. You can sign up for historically. We’ve kept these to clients only, but this year we’re inviting you to tune in to learn how inflation could impact your portfolio and your retirement to learn about what’s going on with market volatility.

We’re going to have myself, Chris Perras, our chief investment officer, James McFarland, Jared Kenney, and Jessica. Part of our job is going to be to break down technical speak into a way that you understand so you can see what’s really going on, what you should tune into, what you should tune out and how it impacts you most importantly. Tune in to the investment summit. It’s down below. You can register for it there. What we have here, we have a 62-year-old couple. Now, if you’re not married, the concepts are still the same here. You don’t have to comment and say, “Troy, do his video for a single person.” We do a lot of videos for single people, but what I’m trying to convey here are the concepts and why planning is important, and how these different aspects are interrelated.

The only thing that would be different if you’re single would be the taxation but we’re not going to talk about taxes on this video. We’re going to do this scenario and talk about taxes next week, and then, of course, you’d have one Social Security check as opposed to two, but also you’re probably spending less money. If you go on a trip, you’re buying one airplane ticket versus two. If you’re going to dinner, you’re buying one dinner versus two, so the concepts, if you’re single or married, still valid. 62-year-old couple, they have $1.5 million saved for retirement, and I had them started out with a spending goal of $85,000 a year. Now, this is going to be a linear spend. We’re not going to reduce it later in life. In this particular instance, 85,000 isn’t a ton of money to be spending annually so we want to keep that up because long-term care easily could cost 10,000 a month, 15,000a month later in life, so when we start to look at spending needs later, I want to account for that possibility, and if it doesn’t happen, there’s more money in the nest egg. If it does happen, at least, we’ve planned to have more money later in life as opposed to creating a spending reduction in what we call the slow-go years which then increases the balance unrealistically. Running it out, I have life expectancy at age 90 for both husband and wife. We have taking Social Security at 62, full retirement age 66 and 10 months, technically 67 let’s call it, and then both spouses deferring to age 70 here.

A couple of things I want to point out here. We’re going to look at the same numbers in different scenarios. We have average return and bad timing. What is bad timing? This is if we have a negative sequence of returns in the first couple of years of retirement. This will be static for this portfolio. Now, this is about a 60% stock portfolio, 55%-60% in that range. The negative returns are -17.94%, let’s call it -18%, and -6.29% in 2022 and 2023. In the early Social Security scenario, because this is taken at 66, full retirement age, age 70, in the early scenario when we take Social Security, the sequence of returns in the near time, the negative sequence of returns has a less of a negative impact on the portfolio balance because we need to withdraw less because we’re taking Social Security.

What we end up seeing though is, over time, living to 90 because we have to take more out of the portfolio later in life, when we look at the current dollars account balance at life expectancy and the bad timing scenario, $1 million versus, if we took it a 67 and bad timing $1.2 million. Here, if we took it at age 70 in the bad timing scenario, we have $1.03 million. Now, you will see it says current dollars and future dollars. What the difference between these two is, in this scenario, you’ll actually have $1.882 million in the portfolio at life expectancy, but in today’s value, in today’s purchasing power, those current dollars are worth $1.009 million. That’s the difference between these two rows.

We see in this particular portfolio spending $85,000 a year with an average return scenario, so this is about 5.5% average return across all three scenarios currently. In the average return or bad timing, the taking at age 67, we end up with more purchasing power compared to age 62 or age 70. If you want to look at that a bit further, hit pause on the screen. Now, the next scenario I want to look at, I’m going to keep the 62, 67, and 70 Social Security examples the same, but I’m going to change the spending level because a lot of you don’t want to spend $85,000 a year. Some of you may want to spend $60,000, some of you may want to spend $100,000.

I want to look real quick at two different spending levels. This is spending $60,000 a year, and of course, it’s adjusting upwards for inflation. One thing I want to note here is that when you have a very, very low need for income from your portfolio, another way to say that is you have a high capacity for risk because stock market volatility doesn’t impact your security nearly as much as someone that has a high need for income, so the lower your need for income, the higher capacity your portfolio’s risk tolerance is or the more capacity it has to take risk.

I’m not saying you should take more risk. I just want to point that out because a lot of times, and we see this a lot with the clients that we help, you can still be okay making bad decisions in retirement. Making bad decisions for a lot of you doesn’t mean you’re going to run out of money. Bad decisions means that you have not optimized your portfolio. You’ve not optimized your values. You’ve left a ton of money on the table most likely. Here’s what we see with $60,000. Whether they take it at 62, 67, or 70, our current dollars, no matter if we have the average return scenario with a bad timing scenario, they’re still okay. $2.4 million versus $2.3 million, $2.6 million versus $2.5 million, and $2.5 million versus $2.3 million. 67 seems to be the best numbers based on portfolio value at life expectancy. The big takeaway here is that if you have a smaller spending need relative to your overall portfolio size, when you take Social Security, it doesn’t matter nearly as much as if you have a larger income need relative to your portfolio size. Now, just because the Social Security decision won’t impact your security, for many of you, I know you still want to optimize your dollars because you want to give to charity. You want to make sure money ismoney is left for the kids. You want to make sure, hey, I could have a very large medical expense later in life, or what if something happens unforeseen? Optimizing your decisions absolutely leads to more money in your accounts over time. We don’t want to be lackadaisical about these decisions, but I do want to point out that if you have a small income need relative to your portfolio size as we see here, you’re probably going to be okay no matter when you take Social Security.

Now, I’ve adjusted the spending need up to $100,000 per year. Many people may feel comfortable at this spending level, even though I’d say it’s a bit much. The reason why people may feel comfortable is because if you turn Social Security on at 62 and you’re getting, let’s say, $40,000, $50,000 a year from Social, then your spending need, your portfolio withdrawal need is still probably around 4%. We’ve been told that the 4% rule is a standard general rule that you should follow. If you follow the channel, you know that the 4% rule is not something that you should follow. Generally speaking, every single person is different, your circumstances, your situation completely different.

I don’t believe in general rules of thumb for retirement. It’s just to get you to click on articles, read books and give people something to talk about. In the real world, it doesn’t necessarily work like that. Here, I want to point out that taking it at 62 in the average return scenario, even spending this large amount of money, we still end up being okay assuming we don’t have a large medical expense later in life, but in the bad timing scenario where we lose money in the beginning years of retirement, this puts us in jeopardy.

Taking it at 67, with the larger spending need, we still are okay in the average return scenario. We’re not too bad here at the bad timing. We at least have some money left at 90. I’d like to see that be a lot higher though, and then over here deferring till 70, we see a big number here on the average return side, but on the bad timing side, not so much because we’re having to take a large amount of money from our portfolio from 62 to 70, but then we turn Social Security on.

In this example, it’s about $90,000 per year if you defer until age 70, husband and wife, but still, because we had to take so much out combined with bad timing, the portfolio gets into such a big hole in the beginning years, it can never recover. Before I move into the investment portfolio and talk about different investment styles and how they can impact your security given the different variables here, quick takeaway for what we just looked at. If you have a low income need relative to your portfolio size, you will probably be okay, no matter when you take Social Security. Taking Social Security at the right time can meet hundreds of thousands of dollars additionally inside your portfolio which does lead to more security for medical expenses and also the ability to give to kids, grandkids, charities, etcetera.

If you have a large income need relative to your portfolio size, then Social Security becomes a lot more critical of a decision at that time of retirement. I would consider a large income need to be somewhere around 5% to 6%, 4.5% to 6.5% let’s say, and a low income need anything around 1%, 2%, 3%. If you’re in that range when you take Social Security, much less of an impact regarding your long-term security. We’re going to keep 62, 67, and 70 as the social securities, but now I’m going to change the portfolio investment style. We’re going to go conservative and look how that impacts everything. Also, I brought the spending level for all three scenarios to $80,000 a year, because I feel that’s a little bit more reasonable number when we’re looking at the average family that has about $1.5 million saved from my experience.

One of the misconceptions a lot of times we have when people come in and they are conservative investors, it’s really important that they hold onto the money that they have. Oftentimes, they want to take Social Security sooner because that makes them feel more secure. The more money we have for rainy days, for down the road, the more secure we feel, when the truth is, again in retirement, the most secure feeling you can have or the most secure element of your retirement is not how much money you have, it’s how much income you have, because that’s what we live on.
It’s a false sense of security a lot of times to leave more money in the accounts and take Social Security sooner because that early Social Security gives you less income over time and less income typically means less security. What I want to look at, conservative investor, this is someone who has a lot of bonds, probably some cash, some CDs, maybe 10%, 20%, maybe 25%, 30% in the stock market over time, that type of allocation, we really can’t expect large returns. We’re probably looking at somewhere in that 3% range over time, which for a lot of you is not going to cut off. For many of you, maybe it’s fine if you have a very, very low income need. In this particular scenario with that portfolio allocation, 62 Social Security, 67, 70, spending $80,000 per year. They have an account balance on an average return basis of almost 700 versus a bad timing of 551. They’ve been very conservative with their portfolio, so the bad timing barely impacted their long-term security. Because they weren’t earning enough interest to really keep up with inflation and maintain that purchasing power and really grow the accounts over time, they still end up in a very marginal situation to say the least simply because if there was some type of major healthcare expense down the road, the savings could be exhausted, and then they’d be living on a Social Security check that was much, much smaller than it otherwise could be because they took it at 62 as opposed to 67 or 70.

Scenario 2, age 67, conservative portfolio again, but still in the bad timing scenario, they still have about $800,000 in purchasing power, 924, the average return. This is probably a bit more comfortable if there’s $300,000, $400,000 of end-of-life care needs. We’re not worried about exhausting those funds. Even if there’s $400,000, $500,000, $600,000, there’s still a little bit of money left there. Comparing to age 70, very similar as far as the average return and the bad timing scenarios with this. The difference would be, of course, 70, we have the same account balances, but we would have a much higher guaranteed lifetime income stream. That means that this person who deferred till 70, even though they have the same account balances is much more secure. I’m going to show you how much secure now.

In this example, if you took Social Security at 67, by the time we were 85, Social Security still would be a really good number because of cost of living adjustments. It gets up to $103,430. Now, if they waited to take Social Security until age 70, by the time they hit 85, the Social Security is a $129,000, so what’s that? An extra $2,000 per month of guaranteed lifetime income every single month. If they live to 90 or 92 or 95, the cost of living adjustments will also be higher because it’s a percentage based on your Social Security income. If the income is higher, the percentage increase is higher.

In our example, here, these two scenarios, they have about the same amount of money in the bank or in their investment portfolio but one person has about $225,000 more per month in guaranteed lifetime income, so this person is more secure. Now, I want to look at a more aggressive portfolio, someone who is comfortable being in the market, and we’re talking about 8% average annual returns here. If you spoke to most experts, 8% per year might be a little bit on the high side of what we expect moving forward. Now, I know historically the market has done X depending on which timeframe you look at, 9%, 10%, 11%, truth of the matter is, this is an unprecedented time. The federal reserve has never supported the markets like they have over the past 10, 15 years in this country, so there really is nothing to compare it to. If we’re going to unwind all that money the federal reserve has put in to the economy and the marketplace, we have to most likely lower returns, but this scenario looks at higher returns, 8% per year. Spending the $80,000 taking Social Security, 62, 67, 70, we see in the average return in the bad timing scenario, what’s really interesting here, is average return 3.7, 2.8. First is taking at 67, 3.8 and 2.8. In this instance, when we have really good returns, even though the negative, the bad timing in this portfolio is -27 and -9. These are pretty bad returns for the first year of retirement while taking out $80,000 from the portfolio. The good news is though, when we look at it and this is in a vacuum, in the real world, we’re not going to hit 8% every single year but if the market does recover and perform well for the rest of your retirement, you’re not nearly as impacted with an aggressive portfolio from that time moving forward, then it’s taking Social Security to 62 versus 67 than if we took it at 70. If we look at the bad timing scenario here, 2.25, 8 million. This is almost what $500,000, $600,000 less than taking it at 62 or 67. That’s interesting to me but when you think about it, if we defer Social Security to 70, we have $1.5 million and we’re spending $80,000 a year, then we lose 27%, so let’s call that another 300,000 or so, we have to take out 80 plus 300,000. Then, we have to take out at 63, 64,65, all the way until 69 until we turn Social Security on.

When we have a very bad year in the first one-two years of retirement, what this tells us is, it may make more sense to take Social Security sooner so you can leave more money in your portfolio and allow that power of compound interest to work its magic over time, as opposed to being very rigid in your decision making and not having the wherewithal or the self-awareness to have a more fluid or dynamic plan in retirement to where, “Hey, life happened, things didn’t go exactly as I planned. I need to adjust my retirement plan now.”
This is a great example if we stick because you’ve read an article that says you should defer Social Security as long as possible but in the real world, when you look at your spending and the investment performance, when we look at the numbers running the analysis, deferring till 70 may not be the best idea in this particular situation. The takeaway here is, if you have a low-income need relative to your portfolio size, you can get away with making a lot of mistakes, but it will still cost you hundreds of thousands of dollars over time. If you have a large income need relative to your portfolio size, you really don’t have any room for error. You need to be consistently making good decisions year after year after year, and this is particularly around the decisions you make with income when you take Social Security and how the portfolio is invested.

Now, next week, we’re going to look at the same situation, different Social Security timings, different income levels with the tax plan. This is just the investment plan of the income plan. This is what we call the first three steps of the Oak Harvest Retirement Process process. We have to have an income and an investment plan absolutely. Once those are in place, now we can start to look at the tax plan and then go back and make adjustments to the income in the investment plan as needed.

The most important thing is to stay connected to the plan. Year after year after year, we’re looking at all of these connected pieces and decisions that you have to make, and our job as retirement specialists is to help you understand the pros and cons, and not just the impact today but the long term impact on your security of making various decisions today. When it comes to the investment style of the portfolio, some of the big takeaways here is if you have a conservative investment portfolio, meaning heavily towards CDs, heavily towards cash, heavily towards bonds. Of course, the bad timing scenarios don’t impact you nearly as much but at the same time, you’re jeopardizing your security if you take Social Security early, because the portfolio won’t grow enough to keep up with your larger income needs later in life. Those needs are larger because your Social Security income is smaller because you’ve taken it sooner.

In all these scenarios we looked at today, if I had to choose one time to take Social Security, just in a vacuum without looking at it year to year, it would probably be full retirement age. It seemed in most of the scenarios, conservative portfolio, aggressive portfolio, bad timing, good timing, to have the most or the highest probability of success. Now, again, in the real world, this is something we would look at every single year. We want to look at it a couple of times a year and make adjustments as needed, have these conversations, what’s working, what’s not working, how do we make better decisions to put you in a better position to succeed? All right, guys. By now, you know the deal. Subscribe to the channel, like the video, and comment down below.