Excellent Data Visualisation - Mortality Statistics Meets Modern Video Technology

Exciting statistics on visual display on BBC4. It indeed is an exciting, visually pleasing and modern video. The presenter, Hans Rosling, a statistician and a guru of data animation, makes numbers look matrix-cool! I thought I was watching a 4-minute magic show. Savour in the power of great data visualisation. Watch the life expectancy and wealth progression of 200 countries in 200 years in 4 minutes.

Smart Systems and Competent Systems

It amazes me how companies won't do the most basic things with their data. About once a quarter the company that rents us our flat solicits us by mail to sell the place. I just recycled a letter from our current broadband provider encouraging us to switch to them as they have better reliability and lower rates than the competition.

Surely there should be a database out there where a simple join between a residential addresses table and a current customers table would result in a mailing list that does not include me. I'm not sure what offends me more, the excess waste this represents not just in felled trees, but in the entire supply chain that delivers me this mail, or the simple incompetence that it represents.

The Economist has an interesting special report this week on Smart Systems. This report portrays a future where the rapidly progressing sensor, wireless communication and power/battery technologies converge to deliver endless data enabling us to analyse and optimise everything. Power grids, water works, and even cows are candidates for this new age of analytics. They could be exciting times for Operations Researcher practitioners. Early benefits will probably come from simple applications and may resemble the traditional benefits from IT and access to information. As a second level, Operations Research will be able to do more sophisticated things with the data, but when I see the examples I mentioned above, it can be possible to lose faith.

Young OR Conference April 2011 - Consultancy Stream

The OR Society is hosting the biennial Young OR conference in the University of Nottingham, United Kingdom, on 5-7 April 2011. I am organising the Consultancy stream, and I am looking for speakers, presenters and of course audience. If you are disregarding the conference because of the word 'young', think again, because the definition of 'young' in this context is <= 10 years in the field of OR. You can find more information for presenters here. Essentially, this is what you need to do:

• a 200 word abstract for the conference programme
• a presentation of max 20 minutes
  

I described the stream as follows:

The consultancy stream aims to attract speakers and audience interested in sharing their experiences in the practical application of Operational Research in a client-consultant setting. The consultant can be internal or external to an organisation. The problem at hand can be simple or complex, technically or organisationally.

The challenges we face as OR consultants are very similar no matter the industry, the organisation or the problem at hand. There are definite gaps between practical application and academic research in OR, but it is still one of the most rewarding jobs. The recommended format would be a case study presentation covering the entire cycle of the project where possible, but presentation creativity is absolutely encouraged.

• How did the problem find you or how did you find the problem? i.e. How was it sold?
• Steps taken to establish your course of action
• OR and non-OR techniques and methodologies used to structure and solve the problem
• How were your findings and recommendations communicated to the stakeholders and decision makers in an effective way?
• How did the client take your recommendations? Did they implement?
• Finally, what do you enjoy most about your job?

Most of all, have fun and meet some fellow Operational Research practitioners.

Please pass on the message. Better yet, please drop me a line to present! As you can see, the stream description is very wide, encompassing all real life applications of OR. You don't have to have 'consultant' in your title, neither does your company or organisation. Come and share your experience and the fun (or pain?) of applying Operational Research in anything from ordinary day to day life to extraordinary situations of, for instance, life and death and taxes. How have you helped with better and more informed decision making?

Oyster Card Optimisation

Transportation is an industry where a lot of Operations Research is practiced. In the following article I would like to share an example of optimisation that I have noticed in the fare pricing system on the London Underground.

Public transportation in London, England has a convenient and efficient means of collecting fares from travellers. Introduced back in 2003, the Oyster Card is the size of a credit card and is pre-loaded with money by the traveller. On each trip they take, the traveller touches the oyster card to a reader, registering their journey with the system which deducts payment from their balance. Each single journey is charged at a different rate depending on the origin zone, destination zone, and time of day.

A daily capping system is in place such that you will never pay, in a day, more than the price of a day-pass covering all of your journeys for the day. For example, in a day where you only travel in zone 1 off-peak your journeys will cost £1.80, £1.80, £1.80, £0.20, £0, £0 and each journey after that is free, as you essentially now have a day-pass on your card when your daily cap has reach at £1.80*3 + £0.20 = £5.60.

A Canadian friend of mine, currently residing in Australia, visited me here in London the other weekend. Knowing the ease, convenience, and price-capping guarantee, I recommended that he get an Oyster Card. He loaded it up with £10 at Heathrow and came into town to drop his bags at my place. After a short jet-lag nap he headed out into the core to see the tourist sights, travelling frequently on the underground. At the end of the day he reported that his Oyster Card credit had run out and that he had needed to top up the balance. This surprised me, so we worked out his journeys and payments:

• Zone 6 (Heathrow) to Zone 1 at Peak - £4.20
• 6 x Zone 1 Off-Peak - £1.80 each

Because he travelled from Zone 6 to Zone 1 at peak, his cap for the day was £14.80 even though had he bought a Zone 1 day-pass at Heathrow he would have only paid £5.60 + £4.20 = £9.80. So the Oyster Card is convenient and comes with a price capping system, but there are holes in that system. In this case it cost him £5.00 which is about an hours work at minimum wage in the UK, so not trivial.

Any individual travelling on a public transportation network wants to perform an optimisation. In this case, they want to minimize their total cost by selecting the most efficient combination of fares to cover all of their journeys. This problem presents itself as a classic optimization problem; Subject to constraints, like the requirement to purchase tickets to cover all journeys, the goal is to minimize total cost, a function of the decisions to buy tickets. An optimisation problem like this can be formulated mathematically and solved by computers using a discipline called integer programming, one of the tools in the Operations Research practitioner's toolbox.

If this problem can be solved by computers, why doesn't the Oyster Card system provide a lowest price guarantee rather than the evidently imperfect price-capping system? Consider for a moment the requirements of the system:

• Daily ridership of around 3 million
• At the end of their journey, users must be told almost instantaneously what the cost was and what their remaining balance is

Optimisation problems of this nature are not always fast, easy, or even possible to solve optimally. The computers of today are fast, but there's plenty still beyond them. The tube system isn't even using the latest technology. I've been told that some Underground components still use punch cards! Every time a customer makes a journey this optimisation must be calculated and that must be done 3 million times a day and that is unfortunately too much.

When an optimisation problem is too big or too complex to solve directly and perfectly, analysts use something called heuristics to come up with near-optimal solutions. There are commonly used methods, but depending on the problem, customised heuristics can be developed, using the unique structure of the problem in question to produce a near-optimal result. That is exactly what the price capping system is; It is a heuristic used to make a good approximation of the lowest price.

There are effectively only two types of tickets in the system: single tickets and day passes. Day passes are the only way to save money. It is rarely worthwhile buying two separate day passes. It follows naturally that a simple rule of thumb for cost optimisation is to compare your daily total of single trips to the price of a day pass covering all those journeys and choose the lower option. The conditions that I list at the start of this paragraph are essential consequences of the structure of the problem, and we can exploit them to arrive at our simple heuristic, the same one that the oyster cards use.

In a future article I hope to look into formulating the optimisation problem of the London Underground and consider alternative heuristics.

Expedia Revenue Management at Check-out or Rule Compliance

We have all been shopping online for something only to be told after making the purchase decision that it is no longer available or no longer available at that price. This often happens when buying flights, as prices can change minute-to-minute and you can be left with a much higher ticket price which makes you abandon your purchase. Disappointment all around.

However, the opposite happens from time to time as well! The price of a London to Seattle flight, when I found it was £649.07 (including all fees). I clicked to start jumping through all the purchase hoops, but after a couple steps into the check-out process, it flagged up, rather alarmingly, as £616.07. That's a 5% decrease in price. (See, I'm not making it up!)

I was pleasantly surprised, of course. But why would they do that?

I've got 2 suspicions.

1. Revenue Management / Yield Management / Consumer Psychology

In the weeks prior to this screen capture, I've been to the site a few times already looking for the exact same flight. Even though I'm not logged in, I'd venture to guess that the site has looked up my cookies and knew that I've been looking for these flights. Therefore, it should know that I was a likely buyer, rather than a window shopper (pc pun intended). I've been at the check-out stage before, but have abandoned the shopping cart eventually. It would be quite logical for the site to entice me with a lower price as a 'pleasant surprise' to finally get me to spill my moola. Not to mention the positive impression it's left with the shopper (look what I'm doing now - free advertising!).

However, is it worth the 5% price drop? How does Expedia decide 5% was the right balance of customer incentive and revenue loss? I was already a willing customer, ready to bite. Isn't it just giving the 5% away for free? In my case, it's difficult to say whether the move has gained my loyalty to Expedia, because I was already a frequent visitor and buyer there. It may have re-enforced my loyalty though. It would be very interesting to analyse a few year's purchase and cart abandonment data of customers where this has happened to, versus a control group. Would we observe a lower purchase completion rate, which would drive a higher lifetime revenue per customer?

2. Airline price adjustment rule compliance

There could exist such a regulatory rule in the online airline pricing world to protect consumers, such that the vendor must notify the buyer of last minute price changes before the final purchase is completed. Now, I don't know if such a rule exists, but it is possible. However, it sounds extremely difficult for the regulators to enforce and monitor compliance.

I personally think it's more the former than the latter. One way to test the real reason behind the price drop could be to see if it's always a 5% decrease. Time to do some more flights window shopping!

P.S. In a previous article where we observed operational inefficiencies at London's Gatwick Airport, we erroneously stated that the airport operator was BAA (British Airports Authority). In fact, BAA was forced to sell Gatwick to please regulators seeking to break a monopoly on UK's airports. Our apologies to BAA. The current owners are Global Infrastructure Partners, who also owns 75% of the London City Airport.

Update:

Responding to two unconstructive comments, one of which was downright rude and was deleted, we thought we would add to this article.

The commenters suggest that Expedia is not a price setter, but just a re-seller making possibility one above unlikely. That said, the question still stands, "What's going on here?". If the prices that Expedia gives you when you search are cached and not live, that seems to be to be a surprising shortcoming. If they are, why offer a lower price to someone who appears to have already made the decision to purchase?

There are probably a number of factors at play that someone from the online travel community could answer.

If I were reselling through Expedia, I would want my price-updating algorithm to give the higher of the two prices at the point of payment, i.e. more profit. Both Expedia and the vendor are motivated to collect a higher price and therefore a higher commission as a percentage of the selling price.

The commenters may be very correct in saying that Expedia doesn't set the price, but merely re-sells at whatever the price the vendor names. That's why we said there were two possibilities, the second being not revenue management. However, if Expedia is not practicing revenue management in this way, they probably should at least experiment with it. Their commission represents a headroom within which they can optimize and the goal, after all, is not to make the greatest profit on each sale, but instead the greatest profit across all possible sales.

Restaurant Systems Dynamics - Influence Diagrams

Systems Dynamics is a discipline that floats about in the management science/management consulting ecosystem. It is genetically related to Systems Thinking, though Systems Thinking contains much more, but no aspect of simulation. The two most important aspects of Systems Dynamics are influence/causal diagrams and continuous simulation. Today I would like to outline an example of the use of influence diagrams to study a simple system, gain strategic insight, and form the basis of a stock and flow continuous simulation.

I was in Paris the other weekend, looking for a restaurant for Sunday lunch. Finding a good restaurant as a tourist is always difficult because tourist restaurants just aren't very good. The restaurants in my neighbourhood in London rely a lot on repeat business and referrals from friends and engage in a repeated interaction with their customers. The restaurants in touristy areas on the other hand get the majority of their business based on location. My local restaurant wants to delivery value for money so that I or my friends will come again. The restaurant in Venice never expects to see me again and is motivated to give me the lowest value for money to maximize profit. We have an example here of repeated and non-repeated games, but this is not an article about game theory.

As regular travellers, we have a strategy for finding the right place. There are a number of aspects to that strategy, but the one I want to highlight today is: Find busy restaurants. We are by no means the only people employing this strategy, as it is clear that busyness should be an indication of quality.

Where is this all going? I'm telling this story because I want to use an influence diagram to study restaurants in general, study touristy restaurants in particular and gain strategic insight from that. Influence diagrams are used to study the interactions in a system, particularly the between key strategic resources. In the case of our restaurants these will be:

• Customers occupying tables
  
• Customers queuing for tables
  
• Perceived restaurant quality
  
• Available customers

Figure 1. Simple Tourist Restaurant Influence Diagram

The make-up of an influence diagram is relatively simple:

• Strategic resources, flows or other system variables
  
• Arrows indicating one influencing another
  
• An indication of a positive influence or negative influence
  
• Optionally indications of re-enforcing and balancing loops

Consider Figure 1 above, the influences shown are as follows:

• As the number of "New Customers Arriving" increases, the number of "Customers Occupying Tables" increases
  
• As the number of "Customers Occupying Tables" increases, the "Perceived Restaurant Quality" increases
  
• As the "Perceived Restaurant Quality" increases, the "New Customers Arriving" increases
  
• As the number of "Customers Occupying Tables" increases, the "Length of Queue for Seating" increases
  
• As the "Length of Queue for Seating" increases people will be discouraged and it will reduce the number of "New Customers Arriving"
  
• As the number of "New Customers Arriving" increases, the number of "Available Customers" decreases
  
• As the number of "Available Customers" decreases, the number of "New Customers Arriving" decreases

Re-enforcing loops can be exploited to achieve exponential growth and profit, but can also cause exponential collapse and bankruptcy. Balancing loops are often related to limited resources which limit what we can achieve, but also serve to mitigate damage.

Loop B1 is a balancing loop: As more customers choose to enter our restaurant, the total number of potential customers is diminished, thus reducing the flow of new customers. This puts a natural limit on our business, the number of potential customers.

Loop B2 is a balancing loop: As more customers arrive, our tables experience a higher and higher occupancy and customers must wait in a queue either for other customers to leave or for dirty tables to be turned over. Here is another resource constraint on our system: capacity.

Loop R1 is a re-enforcing loop: More customers leads to an increased perception of quality which then leads to more customers. This is they key re-enforcing loop that we should study further.

The key strategic conclusion that can be drawn form studying this influence diagram comes out of loop R1, the re-enforcing loop. The consequence of this loop is that full restaurants tend to stay full and empty restaurants tend to stay empty. Given that each restaurant starts empty each day, the key challenge appears to be in first becoming not empty. Easier said than done.

Restaurants and bars have a number of ways of achieving this. The first, but least interesting, is simply good quality. A regular customer base or recommendations in guide books will provide the seed customers from which a full house can grow. Alternatively, we need some other means of getting people in the door. This makes me think of my time in Turkey on the Mediterranean coast. Walking along the waterfront in a tourist town, a restaurant owner offered me a half-priced beer as long as I would sit along the front edge of his balcony. If this makes you think of happy hour there's probably a good reason.

I will admit that the "strategic insights" discussed above with respect to the restaurant industry are not earth moving, profound, or even unexpected. However, this article provides a simple real-world example of a dynamic system, and demonstrates the concept nicely. Had we not already known that full restaurants stay full and empty restaurants stay empty, going through this exercise could have revealed that to us.

The next step would be to design a simulation based on the influence diagram, something that I will endeavour to do in a future article.

What motivates us the most

First let me make clear that I am talking about the motivation in workplace. In personal life it's easy - in first half of our life it's the Sex, in second half it's the Comfort. (So to speak with tongue in cheek)

But the workplace motivation is more intriguing. And that is the area that every OR specialist should always keep in the forefront of their mind - the questions and aspects of human motivation. Here's an excellent animated video derived from the talk of one Dan Pink at RSA. Seems that Mr. Pink also excels in self-motivation, since this lecture is a small masterpiece.
True, these research findings are popping up here and there for the last two decades, at least, and lots of companies are adopting some of those principles, however this short video sums it up in excellent concise way. Enjoy!



However, I personally think that all these findings are missing some essential qualifications. I thinks that it reflects the motivation of people in developed countries, where there is no hunger and war is something nobody really remembers.
To echo the words of Mika Waltari in his book Egyptian Sinuhe, where he describes one lucky country he travels through, "...and the people who knew neither hunger no war, were already in middle age...".
I wonder, how the same research would turned out in war torn Angola, or Iraq.
I suspect that this type of "Make the world a better place" altruism grows best in economically nutritious Petri dish - relatively wealthy society. But what do I know about the poor countries. Maybe they would surprise us the most. The world is changing after all. It's the Internet age now.

One observation I made about the phenomenon of people working in their free time for free. (Linux developers, etc.) First I would liken it to simple hobby-ism. And I think that it indeed has the roots in hobbies. Everybody at some time in their life likes to build some "model airplane" and see it fly. But, and here comes my observation, they would like more to see it soar, than just fly. In other words, people don't mind to work for free on somebody's else project (i.e. Linux), but they prefer to jump on winning bandwagon. The likelihood of overall impact (let's even say world wide impact) is a specific motivation on its own.

It's the Internet age now.

Want to be creative? Don't brainstorm.

I'm sure many of us have had the thought before, "Oh, I wish I were more creative". I have. I'm also sure many, many of us have led or participated in a brainstorming session before. I also have. Apparently, these are both counter productive to being more creative, according to this article. To top it off, apparently since 1958 it has been proven that brainstorming doesn't work. I never knew about this. Did you?



In businesses, one of the common outcomes of operational research work is improving a particular process. We often start with understanding the problem, to mapping the process, and to building a model that reflects the current process. Eventually, to add value to the bottom-line, the model hopefully reveals some insights, and is the tool to test out certain ideas to support any process changes. Personally, it is often a pleasure to be involved from the beginning of problem understanding to the end, managing the recommended process changes, because as an OR consultant, you get to see your work to fruition.

Of course, to succeed in change management, the ideas should come from the stakeholders who live and breathe the process in question, and eventually own the solutions to be implemented. To get ideas from stakeholders for process improvement, the common technique is to gather stakeholders in a room and brainstorm on possible solutions.

Given the information presented in the article, to prepare for the brainstorming session, I take away that we should consider to:

• present the problem to the group before the brainstorming session,
  
• ask them to prepare and think about possible solutions that their colleagues or friends wouldn't have thought of for resolving the issues,
  
• get them back in a room to discuss each other's ideas and prioritise on the ones to investigate feasibility and impact,
• but before they start discussions, get them do some aerobics for 30 minutes if they are somewhat fit (half serious, but wouldn't that be fun?),
• culture them with a youtube video about the weird and cool stuff in other countries (half serious, but wouldn't that lighten up the mood?),
  
• facilitate the session with careful language to not instruct people to be creative
• facilitate the session so the group moves back and forth between a couple topics to be able to take a break from focussing on just one solution
  
• and perhaps not name it a brainstorming session, because it may be the forum that people associate with "get creative...now", which is counter productive, as per the article

Read it if you've got 3 minutes. Let me know what you take away from it that I've missed. (See the instant application?) The main points in the article to help someone to be more creative are:

• Don't tell them to be creative
• Get moving
• Take a break
• Reduce screen time
• Explore other cultures
• Follow a passion
• Ditch the suggestion box
  

What qualifies as a Simulation Model?

A theme that has been running through my career since my Master's project has been the question of measuring complexity in modelling and simulation. When can one proclaim to have built a simulation model and when is one glorifying simple analysis?

In the Operations Research ecosystem the tendency is certainly to inflate. Salesmen, curriculum vitae authors, recruiters and consultancies across the spectrum are all motivated to embellish the work that they do and work that is done. Like any scientific individual I seek to slice through the static, inform myself as to who is doing extraordinary work, and to build myself a framework from which I can safely criticize the inflations of others.

I have been working on a set of rules for separating "models" into models, calculations and simulations. I feel like there is a gaping opportunity here for contribution from complexity, chaos, and other disciplines in Computer Science and Mathematics, but here's what I've put together thus far:

Simulations are models, but not all models are simulations. Calculations are not models.

Models

1. A model is a simplified representation of a system.
2. All models are wrong, but some models are useful

Calculations

1. The result of a calculation can be expressed in a single equation using relatively basic mathematical notation.
2. Where calculations contain an time element, values at different times can be determined in any order without referring to previous values.

Simulations

1. A simulation is a calculation in which one parameter is the simulation clock that increments regularly or irregularly.
2. The outcome of a simulation could not have been determined without the use of the clock.
3. While an initial state is typically defined, an intermediate state at a given time should be difficult or impossible to determine without having run the simulation to that point.
4. Almost any model that involves repeated samples of random numbers should be classified as a simulation.

Consider the following progression of "models" that output an expected total savings:

1. Inputs: Expected total savings.
2. Inputs: Annual savings by year, time-frame of analysis.
3. Inputs: Annual savings per truck per year, number of trucks by year, time-frame of analysis.
4. Inputs: Annual savings per truck per year, current number of customers, number of trucks per customer, annual increase in customers, time-frame of analysis
5. Inputs: Annual savings per truck per year, current number of customers by geographical location, annual increase in customers by geographical location, routing algorithm to determine necessary trucks, time-frame of analysis.
6. Inputs: Annual savings per truck per year, current number of customers by geographical location, distribution of possible growth in customers by geographical location, routing algorithm to determine necessary trucks, time-frame of analysis.

As you can see, complexity builds and eventually passes a threshold where we would accept it as a model. "Model" 4 is still little more than a back of the envelope calculation, but Model 5 takes a quantum leap in complexity with the introduction of the algorithm. Model 5 however I would still not classify s a simulation, because any year could be calculated without having calculated the others. Finally Model 6 introduces a stochastic variable (randomness) that compounds from one year to another and brings us to a proper simulation.

I've seen calculations masquerading as simulations models at a Fortune 500 company both internally and externally. While the result is the same: outcomes determined from data where validity is asserted by the author, I know that Operational Research practitioners reading this will appreciate my desire to classify. At the very least it will help us separate what the MBAs do with spreadsheets from our own work.

I welcome input from others on this topic, as I am only just developing my own theories.

Travel, being an OR consultant, and another blog

Activities on the ThinkOR blog has been a bit thin in the last month or so. Summer has arrived and we have been busy enjoying it as much as we can in London. So far it's been a great half year: Exeter UK, Istanbul, Bursa, Ayvalık, Bergama (Pergamon) TR, Riga LV, Berlin DE, Milan, Venice, Padua, Verona IT, the Algarve PT, New Delhi, Agra, Udaipur IN, Bahrain BH, Malaga ES, Reykjavik IS, and of course Canada and the US. Not bad, eh?

Travel:

To travel this much for leisure (18 countries last year), and to cover as many interesting cities as possible that span the continents, i.e. objectives; to not break the bank, to use as few vacation days as possible (we've only used 9 so far), to avoid anticipated bad weather, to not leave work too early for flights, and to not overdo it to tire ourselves out, i.e. constraints; means that we need an optimised strategy. We travel on weekends and use bank holidays as much as possible. We travel budgetly with lean (polite for 'cheap') airlines like EasyJet and Ryanair flying out after 5pm on a Friday, to trade off between more time in the destination and the cost of 1 extra night of hotel, as well as a peak rate for flights after 5pm. We make a judgment on the trade off between the central location of hotels with the higher cost usually associated. We also need to do our research on the temperature and the likelihood of rain for the cities on our list, and line the cities up with the weekends we would like to travel, but our list is often dictated and changed by the destinations of the airlines and the routes on sale. Our part time job is a travel agent, because it is quite time consuming. However, we usually plan a couple months in a batch process, and don't need to think about it again once it's in the diaries. It's kind of fun planning it, and more fun zipping away every second or third weekend.

Being an OR consultant

I just started a new job at Capgemini Consulting's operational research team. Already did one project with a major consumer product manufacturing and distribution company. Very interesting project, in which I enjoyed working on modelling their supply chain and the cash to cash cycle, and the impact of one seemingly simple decision's impact on the bottom line. This is exactly what OR is for - helping businesses make more informed decisions. The project was quite short and intense. I feel like one of the most important attributes OR people bring to the table in situations like this is what and when you can use averages, what assumptions are ok and what would come back and bite you in the butt. Perfection mostly takes second seat to delivery deadlines. It reminded me of what an advisor told me at uni, "what you learn at school will get applied very little in real life, because businesses never have the time to give to an OR guy to properly figure out the problems and solutions. They want quick answers and they want it now."

Another OR blog

Capgemini has a very cool group of OR people, and they have an OR blog too! Figure it Out. Check it out. Interesting articles on the real life applications of operational research, particularly relevant to UK topics. Of course, I will be writing for them too, as soon as I acclimate a little bit.

P.S. We at ThinkOR are very honoured to be named as one of the favourites in the OR blog world by Maximize Productivity with IE & OR Tools. Thank you very much. It is a real honour. Please let us know any topics you'd like to read about more, and we will try our best to research and write about them.