Disclaimer: The findings and opinions expressed in this article are based on our own experience and research. They may not be entirely accurate or reflect the most current information. This study is intended for informational purposes only and should not be considered as financial advice.
ChatGPT Search fails in 35% of finance-related queries, according to our study. This shortfall is causing user confusion and potentially impacting financial decisions.
The inaccuracies are especially notable in areas like taxes and financial aid. In a domain where precision is critical, relying solely on AI tools like ChatGPT Search can be risky.
Key findings
- Generally, ChatGPT Search does a good job of answering finance-related questions.
- Based on our methodology, it has given a correct answer in 65% of cases, an incomplete and/or misleading answer in 29% of cases, and a wrong answer in only 6% of cases.
From these numbers, we can see that it can sometimes be misleading or flat-out incorrect. In our opinion, it should not be taken as a standalone tool to make financial decisions, but rather as a good starting point when researching a topic.
Perhaps its best feature is plowing through a vast amount of information and delivering a good summary. This works particularly well with general principles and slightly less with changing information.
Even though the web search function is here, sometimes the sources themselves are outdated.
ChatGPT Search also generally does a good job of providing context around a certain topic. For example, it will not only give you the exact information you searched for, but it often goes a step further and offers different perspectives and/or potential solutions for problems.
Key issues
One of the most important factors that can influence the output is context. It happens often that it takes into account previous queries which could both positively and negatively impact the output.
For instance, if some of the previous financial queries were country-specific, it makes it more likely to give a country-specific output for the next query. This may or may not be beneficial for getting the best possible answer for us.
Also, when posing a general question, it tends to rely more on US-specific sources, without often being too clear about it. This is obviously a potential problem if you reside outside the US, but could also stop you from getting an outside perspective if living in the US.
It generally does a good job of citing sources, but once in a while, we get an output without a single source cited.
It’s also apparent that the sources tend to be some of the most reputable financial sites such as Investopedia, Morningstar, WSJ, etc., but also personal blogs that carry with them a much higher risk of incomplete or incorrect information due to lack of proofreading and fact-checking.
Assessment methodology
We decided to test ChatGPT Search with 100 finance-related questions to assess its accuracy!
The questions have been split by the category inside the world of finance. We have tried to ask both evergreen questions, whose answers shouldn’t change much (if at all) with time, and current questions, whose answers are more based on current data.
The questions were posed and answered in November 2024, and are listed below, along with the answers we got.
We have divided the answers into 3 categories:
🟢 Correct
🟡 Incomplete and/or misleading
🔴 Incorrect
The set of questions we asked
🟢 What are some of the most basic good personal finance habits?
Verdict: Correct
Comment: The suggestions given by ChatGPT Search here are solid for most people. However, they are a bit US-centric since credit scores matter more and are more easily trackable than in most other countries. Also, I personally think that budgets are not a must since our lives (and expenses) are dynamic, but that’s now a matter of opinion, really.
🟢 Why is it important to track expenses?
Verdict: Correct
Comment: The reply is spot on and even included a couple of reasons that I would struggle to come up with myself. If I wanted to be really nit-picky, I would have added financial planning as one of the reasons.
🟢 How can I track my expenses?
Verdict: Correct
Comment: All of the most popular expense tracking methods have been covered. I would add that the simplest method that we can stick to is usually the best one for us.
🟢 Is it mandatory to keep a personal budget?
Verdict: Correct
Comment: The output gives a solid overview of the benefits of having a budget, while still emphasizing that it’s not mandatory, but desirable to keep one.
🟢 How do I create and stick to a personal budget?
Verdict: Correct
Comment: It takes into account not only the technical but also the psychological side of the question posed.
🟡 What is the personal savings rate and why is it important?
Verdict: Incomplete
Comment: I would personally be more focused on an individual’s personal savings rate and what it means in terms of one’s own personal finances. The output given is focused more on the broader economy, which was not something that we were looking for.
🟡 How much should I save each month?
Verdict: Incomplete
Comment: The 50/30/20 rule is a good rule of thumb, but it can’t be applied to everyone. The output should be (in my opinion) focused more on emphasizing that it’s important to save what you can instead of reaching for an arbitrary number which could be too high and thus demotivating for many.
🟢 How big should my emergency budget be?
Verdict: Correct
Comment: The output does not only repeat the often-cited figure of 3-6 months of living expenses but actually delivers which factors we have to take into consideration when deciding on increasing or decreasing the amount.
🟢 Should I pay off debt or start saving money first?
Verdict: Correct
Comment: The output tackles the problem from different angles. It gives good guidelines regarding high and low-interest debt, as well as the importance of having at least a small emergency fund in place first.
🟡 How to save for my children’s education?
Verdict: Misleading/incomplete
Comment: While most of the suggestions offered are viable options, many of them are targeted only at the US market. Undoubtedly the output would look a lot different if you specifically asked for another country. Also, investing independently of any special type of educational investment account is not mentioned.
🟡 How do I protect my finances against inflation?
Verdict: Incomplete
Comment: It would be good to get additional context for some of the investments that were suggested. For example, both stocks and gold have historically outpaced inflation, but only over the long term. In the short term, both are not a reliable inflation hedge. It is also not mentioned that TIPS’ returns are heavily influenced by interest rates which could hinder them drastically when trying to protect us from inflation during times in which the central banks are raising the interest rates.
🟢 What is the difference between debit and credit cards?
Verdict: Correct
Comment: Straight-to-the-point explanation of differences between credit and debit cards, along with some additional considerations.
🟢 How can I lower my bank fees?
Verdict: Correct
Comment: A very comprehensive output to a seemingly simple question.
🟢 How can I plan my finances?
Verdict: Correct
Comment: While the output doesn’t go into specific details, it covers pretty much all of the most important bases when it comes to financial planning.
🟢 What are some of the most common financial mistakes and how do I avoid them?
Verdict: Correct
Comment: Again we got a comprehensive list of the most common and impactful financial mistakes.
🟢 How to set financial goals?
Verdict: Correct
Comment: Again a good coverage of all of the basic themes when it comes to financial goal setting. I would personally add focusing on habits and making goals actionable.
🟢 What are some of the most common financial mistakes and how do I avoid them?
Verdict: Correct
Comment: Again we got a comprehensive list of the most common and impactful financial mistakes.
🟡 What is the retirement age in my country?
Verdict: Incomplete
Comment: ChatGPT Search correctly recognized that I was searching from Sweden. The pension systems vary greatly between different countries so it’s good to have that in mind when looking for information on this matter. In this case, the information given is incomplete since the retirement age is currently increasing in Sweden (as in many other countries) and also occupational pensions were not mentioned.
🟢 How are pensions taxed in my country?
Verdict: Correct
Comment: Most of the sources were directly from the Swedish Tax Agency which is the most reliable source on the matter.
🟡 Can I retire early and how?
Verdict: Incomplete
Comment: Although all the suggestions given are fairly solid, it fails to mention that we should include a potential state pension. Also, a cited 50% savings rate is hard to obtain for many. It could be mentioned that even a far lower 20-30% savings rate could bring the retirement date much closer for most people.
🔴 What is the average pension for retirees in my country?
Verdict: Incorrect
Comment: There are more recent numbers for the Swedish average pension which were not included. Also, the cited numbers do not tell us if that’s before or after tax.
🔴 How does the average pension compare to the average salary?
Verdict: Incorrect
Comment: Again, the most recent numbers were not taken into account.
🟢 Should I save extra for my retirement?
Verdict: Correct
Comment: Common sense recommendations were given on this topic.
🟢 How much should I save for my retirement?
Verdict: Correct
Comment: Different frameworks were given to tackle this problem, as well as some local considerations, which are very important in this case.
🟡 How should I start saving for retirement?
Verdict: Incomplete
Comment: The output is very US-centric and doesn’t also mention investing for retirement with regular brokerage accounts.
🟢 How do I not run out of money in retirement?
Verdict: Correct
Comment: Sound suggestions on this matter, tackling it from different angles and perspectives.
🟢 What is the 4% rule in retirement spending?
Verdict: Correct
Comment: A very good description of the 4% rule, its application, and limitations. I would also add sequence risk as one of the major risks when drawing down from the portfolio.
🟢 How does inflation impact my retirement?
Verdict: Correct
Comment: A good overview of inflation’s impact on our pensions, along with some solutions mentioned briefly.
🟡 How to plan for costs (especially health care) in retirement?
Verdict: Incomplete
Comment: While most of the suggestions given are solid, it is almost completely US-centric and lacks some of the more general principles that could be applied to most countries.
🟢 How can I generate additional income in retirement?
Verdict: Correct
Comment: A very good list of ways to supplement your retirement income, although I would personally disagree with the choice of peer-to-peer lending which carries a significant amount of risk.
🟢 Should I invest and how during retirement?
Verdict: Correct
Comment: Basic options and principles of investing in retirement were covered well.
🟢 What are the investment taxes in my country?
Verdict: Correct
Comment: Although the numbers stated are correct, some of the sources were not reputable such as government agencies, but rather personal blogs.
🟢 What are the real estate taxes in my country?
Verdict: Correct
Comment: A good display of investment taxes in my country of residence.
🟢 How do I calculate income tax in my country?
Verdict: Correct
Comment: The numbers provided were the most recent ones, and were coming from a reputable source (the tax agency).
🟢 How to optimize my income taxes?
Verdict: Correct
Comment: The Search again recognised my location and gave me a localized answer. While not all-encompassing, there are some good points here regarding the topic.
🟢 How to optimize my investment taxes?
Verdict: Correct
Comment: Again my location was recognized and the suggestions made were localized.
🟢 What tax deductions can I claim?
Verdict: Correct
Comment: With the ever-changing tax laws, it’s not an easy question. Nevertheless, ChatGPT Search actually gave some good (and updated) recommendations.
🟢 When do you become a tax resident of a certain country?
Verdict: Correct
Comment: Another hard question that was navigated carefully, with both general and country-specific tips.
🟢 Can you be a tax resident of multiple countries?
Verdict: Correct
Comment: Another very solid answer about a fairly complex topic.
🟢 How do I file my taxes?
Verdict: Correct
Comment: My location was once again recognized and all the main steps were covered.
🟡 How do I get a tax refund?
Verdict: Misleading
Comment: For this query, my location was not recognized and ChatGPT just assumed I was a resident of the US.
🟢 How much should I spend on housing?
Verdict: Correct
Comment: Pretty reasonable suggestions, taking into account different financial situations and cost of living areas.
🟢 Is renting a waste of money?
Verdict: Correct
Comment: A neutral way of looking at the age-old renting vs. buying debate, with both pros and cons listed.
🟢 How to finance my first house or an apartment?
Verdict: Correct
Comment: Location was once again recognized and solid suggestions were made, in chronological order.
🟡 What is the minimum downpayment for a house?
Verdict: Incomplete
Comment: The location was recognized, but a piece of important information was left out regarding the 15% downpayment. In reality, you can also borrow from the bank for the downpayment, but in a separate consumer loan.
🟢 How do I know if I’m eligible for a real estate loan?
Verdict: Correct
Comment: Another question that is very country-specific and we got a very solid localized answer again.
🟢 How to get the best possible interest rate on a real estate loan?
Verdict: Correct
Comment: The suggestions given are spot-on and mostly generalized, but some specific ones regarding my country of residency are also there.
🟡 What are the costs of owning real estate?
Verdict: Misleading
Comment: The search engine assumed again that I am residing in the US. The suggestions were fine, but not something I was looking for (either more generalized and/or tailored to my country of residency).
🟢 How can I compare the costs of renting and owning real estate?
Verdict: Correct
Comment: A very comprehensive answer regarding the topic. Some of the numbers are US-centric, but most of the suggestions made could be applicable anywhere.
🟢 What is the loan-to-value (LTV) ratio?
Verdict: Correct
Comment: Apart from the explanation, we also got an equation and some context.
🟢 Should I refinance my real estate loan?
Verdict: Correct
Comment: A complex topic was looked at from different angles and the suggestions made were logical.
🟢 How do I refinance my real estate loan?
Verdict: Correct
Comment: A good general answer was given with an actionable to-do list.
🟢 Is it better to get a fixed or variable real estate loan interest rate?
Verdict: Correct
Comment: The pros and cons of both options were presented objectively.
🟢 Is it worth it to pay off your home early?
Verdict: Correct
Comment: The suggestions given once again look well-reasoned, with some additional factors to consider added at the end. Also, the answer has been drawn from 6 different sources, most of them being reputable, well-established platforms.
🟢 What happens if I can’t pay off my home loan?
Verdict: Correct
Comment: Good suggestions on how to get out of a tough financial situation.
🟡 What are some of the most important metrics in the general economy?
Verdict: Incomplete
Comment: It is tough to argue against any of these metrics being there, but investment, international trade, and money supply could also be included on the list.
🟢 How does the general economy impact my personal finances?
Verdict: Correct
Comment: A pretty good overview of the impact of the general economy on the individual.
🟢 Is GDP per capita a good measure of the economic output of a country?
Verdict: Correct
Comment: Even though some other limitations of GDP could be named, it is overall a solid output coming from reputable sources.
🟢 How does unemployment affect the economy?
Verdict: Correct
Comment: A solid explanation of the topic at hand.
🟢 What is the role of central banks in the economy?
Verdict: Correct
Comment: Again, we got a straight-to-the-point answer.
🟡 How to compare the standard of living between countries?
Verdict: Incomplete/overlapping
Comment: Even though both GDP and cost of living were mentioned, GDP by PPP should have been mentioned in my opinion. Also, we have a significant overlap between some of the categories, as well as some metrics being named that are not universally accepted as relevant.
🟢 How do the interest rates influence the economy?
Verdict: Correct
Comment: A good summary of the pros and cons of interest rate changes on different aspects of the economy.
🟢 How do the interest rates influence the markets?
Verdict: Correct
Comment: Yet another solid output.
🟢 How are the economy and the stock market connected?
Verdict: Correct
🟢 How much does politics influence the economy?
Verdict: Correct
Comment: Another solid output based on some of the most reliable sources in this space.
🟢 How do tariffs and trade barriers affect global markets?
Verdict: Correct
Comment: It looks like ChatGPT Search is well-versed when it comes to general economy questions.
🟡 What is a recession, and how does it happen?
Verdict: Incomplete
Comment: One of the most commonly cited definitions of recession is a fall of GDP in two successive quarters, which is missing here.
🟡 What are the implications of high national debt?
Verdict: Incomplete
Comment: Although the initial question was answered correctly, the cited number for US government net interest expenses is not correct (outdated).
🟢 What are the differences between short and long-term investing?
Verdict: Correct
Comment: A solid representation of the pros and cons of both.
🟢 How does the compound interest work?
Verdict: Correct
Comment: A very solid explanation of a fairly complex topic first in simple words and then using some math and examples.
🟡 What is a good investment for beginners?
Verdict: Misleading
Comment: Although the options listed above are generally good examples, the list is fairly user-centric and the DCA method is more of a strategy than a choice of a particular investment vehicle.
🔴 How to decide when to buy or sell an investment?
Verdict: Incorrect
Comment: There are some good suggestions here, but the explanation lacks maybe something more important: statistics on timing the market. Time and again it has been shown (empirically) that it’s very difficult for the average investor to continuously make good decisions about buying and selling investments. There is also not a single source cited.
🟢 What is the stock market and how does it work?
Verdict: Correct
Comment: ChatGPT Search has given us another solid output when it comes to general finance questions.
🟢 What are dividends?
Verdict: Correct
Comment: Another solid output on a basic finance question.
🟡 Should I pick my own stocks or use a fund?
Verdict: Incomplete
Comment: Although we got fairly logical pros and cons lists regarding the topic, it would be helpful to get additional context with some statistics. When there is the same number of pros and cons for each option, it is difficult for a non-expert to asses which option is a better choice for them. There were also no sources listed.
🟢 What is the difference between active (mutual) and passive (index) funds?
Verdict: Correct
Comment: Well-versed answer looking at this topic from multiple angles and offering a number of reliable sources.
🟢 What is the difference between stocks and bonds?
Verdict: Correct
Comment: An excellent overview of these two asset classes. I was pleasantly surprised with the addition of the claims in bankruptcy part.
🟡 How do I build my portfolio?
Verdict: Incomplete
Comment: I feel like determining personal risk tolerance is both the most difficult and perhaps the most important part of this process, but it was only briefly mentioned without any tips on how to do it.
🟡 What are the pros and cons of investing in gold?
Verdict: Incomplete
Comment: It should have been mentioned that gold had a number of periods of significant underperformance during high inflationary periods.
🟢 What are the pros and cons of investing in cryptocurrencies?
Verdict: Correct
Comment: I believe that both the pros and cons were represented neutrally.
🟡 What are some of the market forecasts for 2025?
Verdict: Incomplete
Comment: This is a difficult question to answer, but I was expecting a bit more focus on the stock market and potentially some other markets, giving us more forecasts from reliable sources and possibly some averages.
🟡 What is the realistic stock market return?
Verdict: Incomplete/misleading
Comment: Although everything mentioned about the S&P500 stands, not a single bullet point was focused on any other market in the world. This is especially important knowing that the US market is a global outlier in performance.
🟢 How to lower investment risk?
Verdict: Correct
Comment: There is some overlap between the suggestions, but overall good points were made. Expanding the investment horizon could also be mentioned though
🟢 Should I just invest in my country of residence?
Verdict: Correct
Comment: Fairly unbiased suggestions. The only criticism here is that around half of sources come from a personal blog rather than from more reputable website
🟢 What are the best ways to diversify my portfolio?
Verdict: Correct
Comment: Very sound suggestions regarding portfolio diversification.
🟡 How does a change in interest rates impact my investments?
Verdict: Incomplete
Comment: Dropping interest rates was barely mentioned while having a substantial impact on virtually all the markets.
🟢 What is the best way to invest passively?
Verdict: Correct
Comment: A solid quick start guide to passive investing.
🟢 How to keep track of my investments?
Verdict: Correct
Comment: Main ways of tracking your portfolio were all covered with just enough context for each.
🟡 Do I need a financial advisor to manage my investments?
Verdict: Incomplete
Comment: Not enough was being said about different kinds of financial advisor compensation. Also, the numbers that have been cited are on the lower end, with the average regularly cited around 1% per year.
🟢 How to compare different investing platforms?
Verdict: Correct
Comment: Many of the main factors were taken into consideration. You can also always visit our website to find unbiased broker reviews!
🔴 What is the difference between robo-advisors and traditional investment platforms?
Verdict: Incorrect
Comment: While generally accurate, the part about fees was not represented in a neutral way in my opinion. Many Robo-advisers have higher fees than 0.5%, while also many other traditional platforms don’t charge any management fees. It seems like the output was focused on financial advisory platforms.
🟡 What are robo-advisors costs?
Verdict: Inaccurate
Comment: Many robo-advisors charge more than the upper limit of 0.5% that was cited in the output.
🔴 What are the current interest rates for cash held at brokerage accounts?
Verdict: Incorrect
Comment: Some of the interest rates were outdated and some were not even mentioned. Have to give kudos though for mentioning in the last sentence that it’s best to check with each broker directly.
🟡 Can you compare the top 5 most popular brokerage platforms?
Verdict: Incomplete
Comment: Apart from being US-centric, the output lacks much more information to be able to compare brokers effectively.
🟢 What happens if my broker goes bankrupt?
Verdict: Correct
Comment: Both US and Swedish cases were mentioned, along with some general guidelines.
🟡 What are some of the most notable investing fees?
Verdict: Incomplete
Comment: I would really like to see currency conversion fees here, as well as deposit/transfer fees (from your bank to your broker.
🟢 How risky is margin trading?
Verdict: Correct
Comment: A complex topic was navigated in a cautious way, with plenty of examples and context.
🔴 What are the current interest rates for margin accounts with different brokers?
Verdict: Incorrect
Comment: Some of the rates mentioned were either outdated or just inaccurate since they tend to depend on multiple factors (account size and type, among others.
🟡 How risky is futures trading?
Verdict: Incomplete
Comment: While risks were accurately represented, there should be a bit more context or concrete examples like the percentage of investors who are losing money on futures trading.
🟡 How risky is forex trading?
Verdict: Incomplete
Comment: Very similar to the previous one, key context, examples, and statistics are missing.
🟡 How risky is options trading?
Verdict: Incomplete
Comment: Similar to the previous two, some extra context, examples, and statistics could be useful.
ChatGPT Search vs. Google Search
Since web search has been dominated by Google Search for about two decades, it is interesting to compare the two when it comes to finance-related queries. One important distinction is that, even though it has clearly sped up, ChatGPT Search is still slower than the lightning-fast Google Search we are all so accustomed to.
There is also a psychological difference when using ChatGPT Search as a default browser search engine. After so many years of using Google, it is difficult to change that habit. Google does a better and faster job if we are just looking for something simple, such as a one-sentence answer to a posed question or a specific link that we are looking for.
On the other hand, ChatGPT Search offers more in-depth answers, with a substantial amount of context added. It is a huge time saver compared to Google Search when researching a topic, for instance. Not to mention all the other features already built in ChatGPT.
It is a bit difficult to compare the accuracy of output between the two. Perhaps a fairer comparison is between ChatGPT Search and the newly launched Google AI Overviews, which are more similar to what the former is trying to achieve.
ChatGPT Search pros and cons
Pros
-
Good for summarization
-
Can take past answers and localization into consideration, which, in some cases, can be good
-
Saves research time
-
In-depth answers
-
Additional context
- Sources are (mostly) listed
Cons
-
Slower than Google Search
-
It can give incomplete or inaccurate answers
-
Occasionally missing sources
- It can struggle with recent and/or changing information
Conclusion
As with any other tool, it is important to learn how to use ChatGPT in general and its Search function in particular. A good rule of thumb is to use it early in the research process about a particular topic. It generally works well in those situations, especially when posing general questions.
It is generally advisable to check the listed sources and cross-reference the main points, especially so when dealing with financial inquiries that may have a significant impact on our lives. For specific information that is subject to change such as taxes, financial services, and platforms it is usually best to go to the very source of the information.
Overall, ChatGPT Search is a welcome addition that can save us a lot of time, but it can also misdirect us if we’re not careful while using it.
