This is a question that many investors (if not all) try to answer every day. In fact, if it were possible, it would be possible to make large returns in a short period of time. However, there is a theory that says no, it is not possible to predict the value of an asset on the stock market, for example. It is known as Efficient Market Theory.
This theory is based on the assumption that all available information is rapidly absorbed by the market by all of its participants and it is impossible to make significantly gains above the market average. This characteristic is the concept of efficient market (FAMA, 1970). Secondly (FAMA, 1991), there are three variants for this hypothesis of efficient markets. The first is the weak hypothesis, which considers that markets absorb only the historical information that is public available. A medium or "semi-strong" hypothesis is based on the principle that asset prices reflect this public information instantaneously. Fama further adds the strong hypothesis, saying that the market instantly reflects even the information considered as insider.
That is, the theory of efficient markets suggests that stock prices follow the Random Walk theory. It states that it is not possible to predict the future on the basis of past data (in line with the theory of efficient markets), that is, it does not mean (for example) that the stock price has increased today, yesterday, or in another period that the price will increase tomorrow as well, because the market works irrationally so the price of a stock will be unpredictable (as well as the movement of a molecule into a fluid).
Let's show in practice how this works by using a decision tree. They are a supervised learning algorithm whose output is If - Then rules that classifies a discrete set of values, given continuous or discrete value inputs. In the figure below we have a classic example of a tree that assists in the decision to play or not given certain conditions like climate and humidity.
In our example we will use a database containing the trading information of the Ibovespa index, from 14.08.2014 to 04.09.2014, every 5 minutes. The available data are Date, Time, Opening, Maximum, Minimum, Last, Volume (contracts), Volume (R$). With this data (1617 records), we created 22 variables to try to predict if, in the next 5 minutes, the index will be up or down.
We run the decision tree in free software Tanagra. After loading the base, we run a χ 2 test to select only the attributes that are really relevant. Here we have our first cut. Of the 22 variables, only 3 remain. The first is if the difference between the maximum and the opening is less than 30 points. The second is whether the volume average of the last two periods is greater than the average of the previous period. Finally, the third measures whether the maximum price of the last period is greater than the maximum of the period prior to it.
Analyzing the result of the decision tree with these three attributes, we see that the chance of knowing if the market will fall or rise in the next 5 minutes is a little bigger than playing a coin. From the confusion matrix, we see that practically the decision tree goes right or wrong at the same frequency.
Notice also the rules that were created. None of them is able to hit more than 62% of the records, with most of them close to 50%.
A curiosity is that in the stock market, a strong consensus among traders is that the traded volume is a strong trend indicator, either for high or low, since it shows that market agents are determined in relation to a specific movement. Of our 3 attributes, 2 are related to the trading volume, showing that the tree was sensitive enough to understand this behavior. Even so, our result is in line with the theory of efficient markets.
Of course, there are more complex methods to try to estimate market movement, but even so, our approach shows that it is a Herculean task that has not yet been completed. The question that remains is: is there any information that is not immediately absorbed by the market, and given this, is possible to use it to predict stocks movement?
The studies continue around the world and if someone finds out, maybe we will never know ;)