Built Like The Brain: Neuromorphic Hardware – Low Power, High Speed

According to this article in Nature:

Superconducting computing chips modelled after neurons can process information faster and more efficiently than the human brain.

We have seen the rise of ML first shift work to GPUs (which were designed for large amounts of linear algebra needed for non-linear editing and video games, making them inadvertently well suited for ML tasks).

We have seen the advent of FPGA and then dedicated hardware for ML, especially Google’s exciting work around Tensor Processing Units.

These are all still based on traditional approaches to computing and essentially classic von Neumann architecture.

Perhaps this new work will lead to a generation of hardware that combines neuroscience, electronic engineering and computer science. There is a level of energy efficiency and speed in the human brain that we are not yet close to matching, even as the raw computing power of ML and distributed systems increases exponentially. Carver Mead’s neuromorphic computing may finally become a practical reality.

The New Bad Apple & iCloud Photo Sync

Synchronizing data across multiple locations or devices is a really hard problem and difficult to get right. There are many variables and determining the single source of truth is often non-trivial. There isn’t even always a right answer.

Understanding that, I have some sympathy for Apple with iCloud (as I do for Google, DropBox and others too.)

But. When you have a service that is being used by billions of consumers, the presence of even small issues or edge cases means that most likely millions of people will have to deal with it.

The good people at TidBITS1 have started a new feature called “Bad Apple” where they talk about annoying and consistent issues folks are having with their Macs. This came about after asking people what problems they were having with their Macs and:

The ensuring conversation spiraled off in numerous directions as various friends and family members griped, kibitzed, and tried to solve each other’s problems. It was fascinating because many of these people were long-time Mac users who had been blindsided by an interface change along the way, and who had thus been frustrated by their Macs ever since.

The first one covers some of the problems with iCloud Photo Library forcing you to re-sync your entire library if you do something apparently harmless like turn it off and back on again.

As I said, these are hard problems often with no easy answers but they need attention. Often this kind of work gets pushed way down the list of priorities in the rush to get new features out the door. Hopefully articles like this will raise the priority internally at Apple.


1 I have been reading TidBITS since the very first issue which I believe I read on one of the comp.mac.* newsgroups. Back in the olden times. (Really. It was 27 years ago.)

On Impatience

In Aikido, there is a technique called shinonage, which literally means “four direction throw.” I have heard it referred to over the years by another name, “the 20 year throw” because they say after about 20 years of practicing it, you are just starting to get the hang of it.

Aikido is like that — very meditative. Nothing comes easy or fast. That’s the whole point in fact. Ōsensei said “progress comes to those who train and train” yet most of us are impatient to a greater or lesser degree.

Impatience can be useful in start-ups which need a sense of urgency to make progress. Like anything, too much can be disastrous.

In his post “Impatience: The Pitfall Of Every Ambitious Person” Darius Foroux talks about the pitfalls of impatience:

And waiting is one of the hardest things in life. But if you take a close look around you, you see many examples of people who waited for the right opportunity.

Foroux talks about the importance of mentors but also highlights his belief in the value of maintaining a daily journal. I have never managed to regularly keep a journal over the years but this post has me seriously considering giving it another try. I just need to find the right app. And then of course, my impatience gets in the way and I find I have moved on to something else…

Steven Sinofsky’s Observations From CES

Steven Sinofsky has been writing some fantastic articles since he left Microsoft and I am an avid reader. You should follow him on Twitter. Go do that now.

He put together a very detailed and enjoyable piece on CES (saving me and countless others from having to endure it ourselves).

He has five key takeaways, which I am not going to spoil for you other than to give you the headlines, because you need to read the whole article:

  • Voice.
  • Electronics for the home.
  • Cars.
  • No wires. 
  • But still too much technology at the endpoint…

Seriously, go read the article. There are lots of great photos too.

Blast From The Past: Apple ][+

I first learned to code when I was 9 on an Apple ][ Europlus that my father bought to do the accounting for his gas station, in 1980. The language was Applesoft BASIC which Microsoft provided to Apple. (Whatever happened to those guys?)

That machine blew my mind. I could make it do what I wanted. That was my first real introduction to what I learned later was software engineering and computer science.

So I was delighted when I saw this awesome nostalgia trip article by Jason Snell over on six colors.

I remember those 5.25″ inch floppy disks so well. The whirring and clanking of the disk drives as you issued the “PR#6” command to get it to boot something off the floppy.

Back then we liked whirring and clanking and computers were super easy to use with the command to “boot from the floppy” being something obvious like “PR#6”.

Like many command line instructions it actually makes a certain amount of sense in context. In this case the context was the floppy drive was connected to a disk controller which was in expansion slot number six, so the command means “prime slot number 6”. Good times. Good times.)

Reading his article also caused me to descend into some Googling that eventually lead me to this  Apple II emulator which is unbelievably cool.

Everything old is new again: Neuroevolution in ML

Great article from Science magazine on a resurgence of research on neuroevolution, the idea of allowing neural networks to mutate and select the best performers, rather than teaching them:

Neuroevolution, a process of mutating and selecting the best neural networks, has previously led to networks that can compose music, control robots, and play the video game Super Mario World. But these were mostly simple neural nets that performed relatively easy tasks or relied on programming tricks to simplify the problems they were trying to solve. “The new results show that—surprisingly—you may actually not need any tricks at all,” says Kenneth Stanley, a computer scientist at Uber and a co-author on all five studies. “That means that complex problems requiring a large network are now accessible to neuroevolution, vastly expanding its potential scope of application.”

Also interesting that the papers cited were published by Uber. A lot of effort going into autonomous driving of course by a number of very well-resourced companies and the impact of all that money and effort is bearing fruit.

The Next Decade: The S Curve of Machine Learning

A little over 20 years ago, I read something from a technology analyst that completely changed the way my early twenties self thought about the world, that has stayed with me ever since.

I had already been using the Internet for about 8 years at that point, initially as an academic network and now in the early days of its commercial use.

I was one of the founders of a technology company and had just moved to San Francisco to set up the U.S. operation for. At the time I thought I was pretty much on the cutting edge because I had an ISDN line in my apartment, which was essentially like a much faster version of dial-up. (It could even bond two channels together for a 128kbps experience.)

The analyst was talking about Yahoo! and in particular its 12 month target stock price. The details of that are lost to time, just like the company itself essentially. (Hello Verizon.)

The comment that he made was that you need to think not of the world as it was then (small numbers of millions of people on the Internet, almost all using 28.8kbps dial-up) but rather the world as it will be. The comment that stuck with me was something to the effect of “Imagine everyone has an always-on high speed Internet connection and you can take that as a given. Now what kind of applications can you build on it?”

Timing is always the hard part but that concept has influenced not only my thinking but the three companies I have started.

I was reminded of it again when listening to Ben Evans talk about S curves and what the future might look like in another 10 years. He touches on mixed reality and crypto-currencies but he spends a good deal of time providing one of the clearest business explanations for machine learning that I have seen. Well worth a watch.

BTW, my favorite line is “every person in this image is a cell in a spreadsheet and the entire building is an Excel file” when talking about automation and referencing this scene from the 1960 Billy Wilder movie The Apartment:

Amazon Attempts to Put The ‘Convenience’ in Convenience Store

The New York Times has a short piece (with lots of photos) on Amazon’s new Go store opening this week is Seattle. The store is opening a year later than Amazon originally said it would but the premise is fascinating.

There are no checkouts or registers. You enter the store using the app, take what you want off the shelves and then just leave. The store detects what products you put in your bag and charges you.

Amazon made a video:

It is apparently smart enough to notice if you put something back and not charge you for it.

Amazon being Amazon, they don’t say much about the technology other than buzzword bingo (“deep learning”, “computer vision”, “sensor fusion”.)

GeekWire did some digging a little over a year ago and had an interesting report that cites some patent applications. One of the tidbits in that piece is a patent suggesting that if the store has difficulty figuring out whether you just picked up a bottle of mustard or a bottle of ketchup, they might use data from your previous purchases to determine which it is more likely to be.

While I am intrigued by the idea, I wonder if it feels (as the NYT reporter mentions) stressful at first when you simple leave a store “without” paying. I get stressed sometimes walking into a supermarket with a bottle of water I bought somewhere else and feel oddly guilty when I am using the self-checkout that I am not paying for the drink, wondering if people think I am shoplifting. Of course that might say more about me than anything else.

The broader adoption of computer vision in retail is going to be a very interesting area to watch, with some interesting cultural changes sure to come as part of it.

Why Does Apple Have so Much Cash?

The insightful Horace Dediu writes some of the easiest to understand pieces on Apple and its economic model. Many people know that Apple is sitting on a large cash pile, over $270 billion (with a “B”) but people ask me sometimes why they don’t spend it (by doing large acquisitions) or give it back to shareholders (dividends or share buybacks.) The reason they don’t do the former is cultural and they do actually do the latter. Horace has put together a great FAQ on all of this.

Going Global With Your Startup

The Y Combinator folks are on a roll with another great post, this time on Going Global With Your Startup.

Kwindla Hultman Kramer from Daily.co highlights 5 key issues he has experienced and heard from other founders when it comes to selling your product or service outside the United States:

  1. Fulfilling international orders is still surprisingly complicated and expensive
  2. If you’re opening an office in a new country, put someone who already knows your company well in charge of that process
  3. Anywhere you have employees, you need an accountant and a lawyer
  4. Work visas are complicated, expensive, and stressful
  5. You’re going to need to get on an airplane sometimes

There is lots more detail and good advice in the post so you should read the whole thing. (I’ll wait…)

I have taken a software company global in both directions – first a company started in Europe (Ireland) that expanded into the United States, and then my current company Qstream, which started here in Boston and then expanded into Europe (Ireland and the UK.)

I have no experience with #1 because my companies have never made physical products. (Technically I guess that is not quite true because my first company was long enough ago that we originally shipped the product on 3.5″ floppy disks and later on CDs. Good times. Good times.)

However, I have dealt with #2 through #5 extensively and strongly agree with the recommendations. I have spoken to groups of entrepreneurs and individuals in the past about going global and you can definitely short-circuit learning a lot of lessons the hard way by talking to someone who has done it.

It is still remarkably complicated to set-up and run a business in multiple locations, particularly for a start-up that is resource constrained. You also need people who know the local culture, customs and laws. If you can get the balance right (don’t rush to do it, take it slow, one country at a time) then it can be a big competitive advantage.