The Need for New Data

Following the markets used to mean gathering data from a few main sources. Investors could keep themselves updated by reading the Wall Street Journal daily. Analysts and financial managers would read financial statements and SEC filings. Some technical analysts tried to glean additional market information by locating patterns in prices. Later, media outlets specializing in financial news, such as CNBC and Bloomberg, started to make it even easier to access financial information.

The Internet made it easy to find an enormous volume of financial data for free at the click of a button. Data availability has been great for investors who want to research companies and access news, financial statements, and SEC filings. On the other hand, the availability of data and the growth of automated, electronic trading made it increasingly difficult for any investor looking to profit by trading on a particular news item or event. The ease of access to this data essentially made the value of that data fall to zero. As a result, many professional traders and wealth managers started to look for alternative data sources.

Finding New Data Sources

Hedge funds, wealth managers, and other professional traders are now exploring alternative data sources to help generate the higher returns their clients demand. A recent study by Greenwich Associates reported that investors are spending $183 million per year on alternative data sources. JP Morgan estimates that when hardware and analytic costs are combined with the actual cost of the data that the market for alternative financial data is closer to $2 billion. Another study, by Tabb Group, suggests that spending on alternative data could double over the next five years.

Some sources of alternative data are more useful for analyzing specific types of investments, so it is important to find the most appropriate data to gather additional information about a company. These are some of the kinds of data available and the ways investors are using them to forecast financial performance.

· Satellite and drone data — Imaging from satellites and drones can provide information about anything from the number of cars in a retail store parking lot to the efficiency of a solar panel.

· Web, app, and social media data — This is some of the most useful data about consumer attitudes and behavior. It can include search trends, data mining for specific words or phrases on social media accounts, and how much time a user spends on a given website or app.

· Weather data — Weather can impact shopping behavior, foot traffic, and even consumer preferences for certain types of goods. Forecasting weather can help in forecasting these activities.

· Location and foot traffic data — Changes in foot traffic can indicate future changes in revenue and profitability for retail and restaurant locations.

· Credit card and alternative credit transaction data — This data can give an aggregate overview of how much consumers are spending and what they are buying.

· Local prices — Aggregating local prices from around the country can give insight into inflation and forecast interest rates and consumer spending.

· Sentiment data — Sentiment measures can help to forecast future consumer spending.

Leveraging New Data Sources

The problem with all of this data is that there is no way a human can process the enormous amount of financial information generated every day. Alternative data sources only add to the already overwhelming task of data analysis. By the time a person could input, model, and analyze all of this data, any opportunities for investment would be gone. This is where artificial intelligence (AI) can help by collecting, analyzing, and modeling the data far more efficiently than a room full of human analysts. AI can quickly find patterns within enormous data sets.

Once the patterns emerge from the data, investors can use this information to model potential market outcomes and investment strategies. Using the tools of AI and machine learning, investors, analysts, and wealth managers can develop far more complex forecasts than ever before to increase their rate of return on investment. The truth is that without the help of AI, trying to incorporate all of these new data sources into a model would not be helpful because analyzing and recognizing complex patterns would take too long. As the amount of available data continues to exponentially increase, the need for technology capable of sifting through the data becomes increasingly necessary for investment decisions. It’s not the data alone that is creating value, it is the dynamic models and forecasts the data allows investors to find new insights and develop better strategies.

AI, Machine Learning, and the Future of Big Data

Alternative data, AI, and machine learning are not just a Fin-tech fad. At its 2017 macro quantitative and derivatives conference, JPMorgan surveyed 239 investors about the future of big data and machine learning. 94% of respondents believed that big data and machine learning would become important for all investors, and 23% said they expected machine learning and big data to start a revolution in finance. Respondents were split about whether or not alternative data and technologies would make traditional forms of financial data obsolete, but the point is that the future of finance is in learning to leverage the enormous amount of data available to investors to make smarter and more profitable decisions.