In last post I wrote why it used to be hard to profit from developing algorithms. This is starting to change due to 3 main factors: Distribution Channels, Open Markets and Technological breakthroughs.
1. Markets -The rise of truly efficient, open, real time markets such as Google Ad Bidding.
2. Technology -True linear correlation of computing costs (Power, CPU, Memory) to dollars spent, thanks to the VMWARE virtualization revolution.
3. Distribution – Web Services such as Amazon Elastic Computing Cloud allow algorithm writers to make them available to the world at zero upfront cost.
Let’s look at them in more detail.
In a virtual market where large sums of money are changing hands all the time, a fast, superior algorithm can be translated into money very quickly. Imagine that you have a faster way to choose when to exchange Dollars for Euros. At any point there are multiple offers to sell and buy Dollars at different rates. Smart algorithm that finds the best deals, at the shortest time, would beat the competition and achieve higher revenue. The problem is that in order to play this “game” one needs huge amount of money to get started. Since the margins are really really low, one need tons of cash to start playing.
New markets are opening up which have a much lower entry level. Assume that you have an algorithm to match the best Google search keywords to what people are actually looking for in Amazon. Now it becomes very easy to profit.
Google puts the search words up for sale in open bidding.
Let’s say your algorithm suggested the word “Soup” would lead people to buy “Seinfeld” DVD.
The algorithm recognized that Amazon will pay 7$ for each DVD sold as a result of the recommendation.
Purchsing clicks from Google is 0.1$ per click.If the algorithm predicts that at least 1 of 70 people who searched for the word “Soup” would buy the DVD, a profit can be made.
Unlike Foreign Exchange markets, Amazon and Google give the small player an even ground to compete, since the information is open and entry fee minimal. All you need is an algorithm, 1000$ for a Linux server and internet access.
Spare CPU cycles used to be a non issue. 90% of servers used 10-20% of their CPU resources. There was no point in optimizing software since there was almost always spare hardware power. On new VMWARE platforms this is no longer true. One physical server can host 25 virtual servers, which means there is strong competition for each CPU cycle. If your algorithm is twice as good, it would cost half the money to run. The binary step of 1 Intel server has been broken (I read this insight on another blog, can’t seem to find to link now ).
Power is the other side of the equation. Power consumption is responsible for 20-50% of the cost in modern computing. A more efficient algorithm usually means lower consumption of power, so it is worth an effort to make create more efficient algorithms.
The new Amazon EC2 service allows anyone to rent a server for 0.10$ per hour. It also allows anyone to have 500 servers for an hour for 50$, or one server for 500 hours for 50$.
The great thing is that the customer does not have to make any commitment at all in advance.
The implications are wonderful. The poor, brilliant researcher can now start her own web based service that provides smart algorithm at a very low cost. Even if she needs large amount of servers, Amazon will supply them on demand, only if she has more customers and demand.
Cost wise, Elastic Computing means you pay for each CPU cycle you use, so it makes great sense to invest in enhanced software, as the relationship is instant and clear.
In addition, there is no need for our researcher to build a complete system and service to start earning income. Platforms such as StrikeIron supply a market place for people to sell core algorithms as a service, that other can use in their own system.
In summary, the open & efficient markets, coupled with elastic computing and virtualization, now allow justice to be made in the computer science world :). Just when it seemed the Internet is only focused on who-can-grow-more-virtual-pets-in-virtual-social-network, we can use it to invent cool new services and make them available to everyone. One can only hope the new algorithms will not center on the problem of growing more virtual pets in automated way …