Layering or Why Nothing Works

Having been involved with computers for 50 years now I have certainly seen some changes. As machines have become faster and memory cheaper the accessibility of increasingly powerful capabilities to average programmers has improved. But the results are not always much of an improvement — in my jaundiced view things are prettier but flakier than ever.

A couple of examples — in my retirement I dabble in stock trading as a way to earn scotch money… when there is a market stable enough to want to trade in, anyhow. I have bought a couple of analytical tools to help me, tools that are fully buzzword compliant. Problem is that funny things keep happening — printing goes all to hell and the formatting routines just ignore the settings, data paths change on their own, complex rules about what stocks to target or dump trigger on symbols with no data (see ‘path’ above). The standard vendor response is to uninstall all of the pieces and reinstall — especially the .net, .net2, .net3 libraries and so forth. The vendor has a nice tool and mostly it is reliable, but just mostly. My suspicions are that the just in time compilations these libraries depend on are less deterministic than they should be — as there is no real memory isolation the end results are probably influenced by factors outside the application. So ‘reproducing’ the problem is probably impossible.

Similar things happen with a systems management application I deployed on my server — it should make patch management easier, but no, I would spend my time chasing internal problems in this incredibly narcissistic application — over night it can log hundreds of errors as its pieces fail to work smoothly together. On the ‘right’ machine it is probably a nice tool but in a small shop there is just no time to sort out its problems. An uninstall is running as I write this.

I guess I am getting cranky in my old age — finding bloated, flaky applications and tools ever harder to tolerate. Had the latest vendor OS and office suite on one machine, pulled both as too many things were harder to do than before. Glad I am technically capable of reconfiguring stuff when it doesn’t behave. Pity it is getting to be a survival skill.

What I suspect is happening is that in too many circles the analytical technique of divide and conqueor has been applied as a general design tool. Methodologies are slavishly applied by implementors who now see that ‘we have always done it this way’. Layers of complexity are built up that should work but don’t because of linkages and dependencies that the simplifying assumptions just exclude [the terms of reference are drafted to exclude societal consequences]. Things are blithly used with no understanding at all of how they work — or fail. So applications dont work reliably, financial markets suddenly break down when factors outside the scope of the carefully designed instrument change.

The root of the problem is that we have embraced complexity as an alternative to the hard work of understanding in a general way what we are doing. So unintended consequences become the norm and things just collapse. Too many semi-skilled specialists and no where near enough generalists. And the big picture is getting fuzzier.


Greed and Market Chaos and Canadian Elections

While I am dismayed at the continual chaos in financial markets, it is interesting to note that in the US, at least, this chaos is traceable to the systematic dismantling of regulations designed to protect the investor. There has been a mantra chanted by the ‘suits’ that unrestricted free markets are the universal solution to all problems. The politicians here in Canada still chant this as they systematicly sell off strategic assets — turning this once strong economy into a client state. All while telling us that we will be more competitive and better off. That was what the US voters were told while special interests pushed for the dismantling of regulations that ended up in the chaos we are all enduring.

Why is it that when politicians get a bad idea into their heads it is impossible to shake it, no matter how devastating the real world experiences are? Could it be that what they really want is to return to a feudal society where a wealth few can do what they want and the rest of us, no longer a comfortable middle class, but starving peasants labor to pay for their elaborate palaces and huddle in fear that we will be noticed? It is interesting that the consequence of all the deregulation in the US was a concentration of wealth — fewer people got more and the rest got less. It seems to happen everywhere the disease spreads — guess the politicians and their golfing buddies are expecting to stay with the ‘haves’.

Deregulation of utilities was an idea originally pushed by Enron — let the market take care of supply and everything will be rosy. Turned out they were manipulating the market to maximize their own profits. But that was not enough to cover the losses from their off-balance sheet deals that eventually brought them crashing down. In the US this idea spread and the results were always shortages and cost increases — everyone is net worse off. But re-regulating the utilities is just not mentioned.

Here in Ontario, after dismantling the provincial utility and restructuring it into a group of ‘independent’ companies a similar curve is being followed — costs are going up, money is poured into hairbrained schemes to make green electricity by devastating the rural landscape with huge wind turbines — despite the European experience that while wind farms might contribute, they are no substitute for coal or nuclear generation. And the experience of others just doesn’t apply…

My fear is that the only way this will ever be addressed is by a general economic collapse and social restructuring. Collectively, we have turned over the reins of power to the greedy and they are stealing for all they are worth. And we are all net worse off. Unfortunately, NONE of the ABOVE is not on the ballot in either the US or Canada.