Comments on: M5_htm Computer Inspired by Hawkins Theories

I like your blog, good luck with the lotto
http://mikecane.wordpress.com/2007/04/27/you-can-now-play-with-a-hawkins-machine/

Tech Center Labs

Gary

Tech Center Labs

Let me know how the M5_NN does, even a tiny bit above random could pay of in the long run.

Gary

Tech Center Labs

http://money.cnn.com/2007/04/27/magazines/business2/sony_playstation.biz2/index.htm?postversion=2007042716

And come to NYC's Chinatown. They are gambling mad over there! OTB, scratch-offs, and draws. (There are also hidden gambling dens, of course.)

For your education, btw, go read The Eudamonic Pie and The Predictors:

http://www.thomasbass.com/work3.htm

http://www.thomasbass.com/work2.htm

OMG! Links that *aren't* to *my* blog!!

Purely random? I doubt it.

Repeating:

For your education, btw, go read The Eudamonic Pie and The Predictors:

http://www.thomasbass.com/work3.htm

http://www.thomasbass.com/work2.htm

Additional: There are *many* Wall Street firms that use such systems on that lottery called the stock market. What, you *really* think they go by earnings, leadership, products, market share? Puhleeze! I've been there on the inside on such a project, and I've read about it too.

I'll stick with this XP version until you make a case for moving to Vista!

That's even better numerology than lottery number picking or TA!

Anyone eejit (credit Cane please) that is stupid enough to flush their hard earned $$$ down the toilet on lotteries and related fools games would jump at the opportunity. Two hundred bucks for a Palm handheld and $30 of software, for a step up on mega-millions. It's a no brainer. Before the foos got enough feedback to realize that they all can't win, these guys would have sold a 100 million units.

Pat Horne

Might sell well to naive Venture Capital though!

Giggle.

You're starting to sound like one of those socialist CNN news drones commenting on the war, mingled together with Hollywood's latest star, with a carbon footprint the size of Kansas, telling you and I to only use one sheet of toilet paper today. :-D

Yesterday my wife delivered twins. To make a long story short she was rushed into emergency surgery and had the 2 month premature girls taken out. Pardon the details, but her placenta had separated from her uterus requiring the emergency C section. I stood in NICU yesterday with a doc who explained to me that had a few more minutes transpired she would have bled to death and both babies would have "suffocated" with oxygen deprivation. After she left I stood among the dozens of computers and medical tech equipment between both 3# girls sustained by technology and thanked God to live in the tech generation. Only 50 years ago, I would right now be pondering life without my wife of 16 years and two babies. A truly sobering thought.

Remember, Terrell Owens and Dennis Rodman are household names, not because of their play, but because of a mainstream media obsession with "shock extremism". My point is that "negative extremism" always trumps common occurrence in the media money machine that feeds us. We must understand that what "they" report may contain bits and pieces of fact, but is in no way the truth.

Don't let the few bad apples in the future of technological revelation dampen the fact that untold bazillions will use these things to improve human existence. The human problem has never been the technology, but the condition of the human heart.

Although, oscillations occur, positive use of technology will dominate the negative use over the coming ages. I say go for it Jeff!

Pat Horne

To LiveFaith,
Glad everything worked out for the wife and kids.
I am all in favor of science and technology. I believe it will eventually lead to a society of leisure and self-betterment. And it will allow people their whole lives to focus on their souls and stop focusing on being wild animals trying to survive in a world were resources are limited, money is king, backstabbing and betrayal a requirement to survive and where work is mandatory or else society dies.

HOWEVER,
Computers, robots and science without AI is OK. My fear is that AI beings will, WITHOUT A DOUBT, evolve (by us first, then on their own) and we will have to then granted them "human rights" and then end up competing against them, and we can't. We need machines to work for us so we don't have to work. Numenta is the beginning of something that has great dangers hidden within. But then again, its human nature to reveal what is unknown. I can only hope we never lets AI computers evolve enough .... BAH... Im stopping typing because stating my hope is irrelevant, its hopeless, I know out nature and AI beings will be among us is a very complex near human form within decades.

In the mean time, I wants a Palm TX with many hardware and OS enhancements.

LiveFaith, me and you will likely be dead by the time Numenta announces the first Terminator. Realize that the beast you are feeding today will be the burden of your newborns when they are adults.

I really hope the thing is pocketable.

Nah, I have to admit I didn't run out and buy On Intelligence.

However, other than reems of gushing and naive Me-Too Media articles, this:

-- http://spectrum.ieee.org/apr07/4982

crossed my path giving me the gist of what the book was about anyway.

Ya want 30-year-old AI/Image Extraction? How about iterative image analysis using Bayes Theory:

-- http://www.google.com/search?num=20&hl=en&safe=off&hs=w0R&q=iterative+zucker+bayes&btnG=Search

Or maybe you want hierarchical image/object operations?

-- http://www.google.com/search?num=20&hl=en&safe=off&q=pyramid+algorithm&btnG=Search

Or perhaps another that is sort of...you know...RIGHT on target?

-- http://www.google.com/search?num=20&hl=en&safe=off&q=quadtree+samet&btnG=Search

Or perhaps you want iterative hierarchical bayesian functionality?

-- http://www.google.com/search?num=20&hl=en&safe=off&q=iterative+bayes+hierarchical

Here's possibly the originals:

-- http://www.google.com/search?hl=en&q=rosenfeld+bayes+image&btnG=Google+Search

Perhaps you're simply interested in Machine Learning?

-- http://www.google.com/search?num=20&hl=en&safe=off&q=genetic+algorithm&btnG=Search

What the Hell, let's go for Hierarchical Machine Learning!

-- http://www.google.com/search?num=20&hl=en&safe=off&q=hierarchical+genetic+algorithm&btnG=Search

In fact, just about any year of IEEE Pattern Analysis and Machine Intelligence is a good reference. Or any of a number of other scholarly journals, for that matter.

As you can see (I hope), there are knowledge benefits to staying til degree completion in graduate school.

That is to say, fool me once, shame on you; fool me twice, shame on me - I don't need to read some rich uneducated kid's thoughts ==in detail== to form an opinion of them, thanks anyway - the Me-Too Media gushes and the Spectrum article suffice to tell all one needs to know.

BTW, since you didn't notice apparently, those were not random Google searches.

=======

Note - I totally respect Hawkins' money - wish I had 1/100th of it - no kidding. Money for SURE is an Enabler Supreme. But that literally is all I respect about Hawkins; sorry for not being a gushing Fan Boy (I don't consider the Palm Pilot nor the Treo as the product of genius, can ya believe that!? I'm weird I guess...).

Considering the tech of the time and the relatively small budget Palm Computing had to work with, the Pilot 1000 hit the ball out of the park in nearly every way (a backlight was really the only notable absence on the original Pilot).

The Palm V was a nice iconic design that was all Ideo and had nothing whatsoever to do with Hawkins. The Treo 600, even in 2003, was an ugly-looking brick and was/is (IMO) entirely overrated as a smartphone formfactor.

For every one of Hawkins' successes (Pilot, Treo 600) there have been an equal amount of flops (Zoomer, Springboard).

That said, I think it's been long enough since he was really hands-on in the mobile computing world that his next release is bound to be a smashing success or an unmitigated disaster. And I STILL think the LifeDrive was a very, very cheap/lazy toe in the waters of this "larger than a PDA, smaller than a tablet" style category.

P.S. For the record, I am going out on a limb and predicting that Hawkins' STB hardware will likely resemble a cross beween this (much less bulky, of course):
http://www.blakespot.com/nino/images/ml3.jpg

and this (with a larger screen and fewer hard buttons, of course):
http://tinyurl.com/2vfk9r

Pilot 1000-->Pilot 5000-->PalmPilot Pro-->IIIe-->Vx-->m505-->T|T-->T|T2-->T|C-->T|T3-->T|T5-->TX-->Treo 700P

Yes, if possible I always turn off any censorship attempts by others for what =I= am doing, ESPECIALLY when that censorship is automated.

Don't you?

You should apply for a job at Palm. Your Palmsource experience with OS 4/5 would definitely be an asset in getting Palm over Linux right.

Are there even any openings left anymore?

Palm can't afford me.

Humble much? :P

Ivan

Take a look at the Numenta white paper at http://www.numenta.com/Numenta_HTM_Concepts.pdf It gets off on the wrong foot almost immediately: "It has been known for over 25 years that the neocortex works on a common algorithm" Then go read Pinker's popularization of the research done on the brain in the last 25 years.

The common-algorithm, neocortex-based approach to AI was originated back in the 50s by Minsky. It played out to a dead end before the 70s, although that dead end wasn't recognized until Winograd wrote his anti-AI rants in the late 80s.

Minsky tried to do this at the cellular level and failed. At first the thought was that there wasn't sufficient computer power available, but by 1985, it was apparent that the computers that were failing to model the brain were more powerful than the brains they were trying to model and doing a very poor job of modeling it. The key failure was Danny Hillis' Thinking Machine which failed to even reproduce the behavior of small organisms.

All Hawkins has done is take Minsky's original approach up one level of abstraction. It's still hierarchical, it's still centralized, and it's ignored the lesson from the robotics community about distributed AI.

To see the direction that Hawkins should be going in, take a look at the DARPA grand challenge that was won by the Stanford entry a year ago. In particular, compare and contrast the Stanford winning entry to the CMU losing entry. You will find the clues to Hawkins failure there, if you're at all familiar with the centipede.

May You Live in Interesting Times

All Hawkins has done is take Minsky's original approach up one level of abstraction. It's still hierarchical, it's still centralized, and it's ignored the lesson from the robotics community about distributed AI.

As I said, it's been 20 years for me since I was eating up stuff like The Society of Mind, but I don't recall Minsky having much interest in identifying or reproducing the physical architecture of the brain--he was all about reproducing intelligent behavior using simple cellular models. Hawkins argues this is both the wrong goal and the wrong method. Furthermore, while he doesn't take on Minsky by name, he does indentify physical structures in the neocortex that Minsky's and other classical neural nets lacked and suggests that they must be central to any theory of intelligence (for example, back-propagation: the cortex is designed to feed much more information back down the hierarchy than upward).

Perhaps Hawkins is not up on the techniques used to pilot autonomous robotic vehicles in the DARPA Grand Challenge--I couldn't say. But saying that all he's done is take Minsky's original approach up one level of abstraction doesn't sound right to me from my own reading of On Intelligence. Maybe you skipped chapter 2?

At least we should be able to agree on this: Hawkins lists eleven testable predictions at the end of the book, so his hypothesis is experimentally falsifiable. If he's "not even wrong" as you say, it shouldn't take long to prove it.

David Beers
Pikesoft Mobile Computing
www.pikesoft.com/blog

Popper was wrong about falsifiability

Yeah, yeah, yeah... Popper, Lakatos, Kuhn, Feyerabend... protective belts, research programmes, paradigms, methodological anarchy... so, we both took undergraduate philosophy of science. Ultimately, just like with the DARPA challenges, it comes down to which technology wins the race. CMU trots out an "ad hoc" modification next year and loses again. As even your friend Lakatos argued, the issue comes down to who has the most fruitful research program (and therefore gets the most fruit).

Hawkins is risking his fortune and as you say, he's set up for a big personal fall (in wealth) if he's pig-headedly wrong. That's what makes good theater of this, and it's a big part of why people are even talking about it. For science, more so even for technology, none of that matters, though. What matters is whether NuPIC (7.0, not 1.0) can make predictions of a type and quality that other technologies can not. You can predict failure--this field is FULL of people making categorical proclamations like yours--but we won't really know until we see who starts winning the races (i.e. the big commercial contracts). Sit back and enjoy the race, man!

As for me, I just love the fact that he's bringing some Silicon Valley style to the arena, what with the developer program and published SDK. Peer review is great, but where the rubber really hits the road is when successful products developed using your theory. Hawkins' democratizing approach to the research is going to make it all very clear: we'll either see them or we won't.

David Beers
Pikesoft Mobile Computing
www.pikesoft.com/blog

I wasn't suggesting that it's something in the water. I also wasn't saying Numenta's approach is necessarily without precedent. Least of all am I saying that any number of entrepreneurial failures invalidate the Silicon Valley model. The "Silicon Valley style" is all about failing. Over and over and over again! As any PIC reader should have learned by now, the place is really a vast field of waist-deep intellectual and entrepreneurial carnage--deep enough to fertilize self-sustaining rounds of still more carnage. That's what makes it great! :-)

David Beers
Pikesoft Mobile Computing
www.pikesoft.com/blog

The senses are integral to what intelligence really is. The brain researchers have figured this out, but nobody's told Hawkins, or he hasn't listened to them.

I'm not sure that's really fair. He seems pretty preoccupied with the fact that for every fiber feeding information to the cortex there are ten feeding it back toward the senses. Part of his frustration with AI is that it seems to have overlooked the role of feedback to the senses (as well as internal to the cortex). If the senses are getting all that information fed back to them then Hawkins would say (in my interpretation) that they are part of the process of creating these "invariant representations" (predictive models), not just passive transmitter/receivers. The reason I'd guess you don't see this in NuPIC is that I just don't think Hawkins regards it as Numenta's job to figure out what "senses" are worth integrating with NuPIC. He wants his developer ecosystem to try all kinds of sensory systems, including ones that are distributed and don't map to the senses that humans have. That may very well be a flaw of the business model: if the senses really are more than dumb transmitter/receivers and are part of the memory prediction framework itself, then just hanging a video camera off this thing isn't going to work very well. I'd expect sensor development would be part of the SDK, and I'm not aware that that is the case. I do know that the research release of NuPIC currently (and by their own admission) is missing some of the criteria that Hawkins identifies as required for HTM, so perhaps this is just one of the pieces they haven't built out. Obviously, the demos he did last year didn't even hint at feedback to the sensory system.

As long as we're on the philosophy of science, it is my strong belief that intelligence isn't subjectable to reductionism. I'm not mystical and I don't think it's an emergent property of complexity. I also think that the centipedes have demonstrated that very well. But I also don't think it's a sum-of-parts problem and I think that's why AI has put so much energy into so much research with so little to show.

That's a very interesting set of statements, but I'm not sure I follow you. By saying you're not a mystic I assume you mean that you don't think that mind or intelligence is some fundamental substance in and of itself. By saying intelligence isn't subject to reductionism I take it you mean we aren't going to boil it down to what individual neurons or other fundamental building blocks are doing. And when you say you don't think it's an emergent property or a sum-of-parts problem, it sounds like you don't think that non-reductionist "holistic" explanations work either. I'm genuinely curious (though increasingly aware we're getting OT): where does that leave you?

Anyway, really fun (and distracting) discussion! Maybe better to take off-line now.

David Beers
Pikesoft Mobile Computing
www.pikesoft.com/blog

I could be wrong--and Hawkins' sometimes watered-down popularization of his own theory suggests I *really* could be wrong--but I don't think HTM is the "big centralized approach" that you describe. I'd have to go back and reread his stuff, maybe look at the SDK, to satisfy myself about this, but the focus on feedback is a pretty good clue. If an HTM makes predictions based on a dynamic competitive process among rival invariant representations (and I think it does) then I'd call that distributed intelligence. To use an analogy from my former profession, an HTM forms predictions a little like the futures market does, through competition among predictions made by autonomous agents operating on incomplete information with no centralized "authority" directing the process.

An example from biology is the way termite colonies create mounds with internal structures that use sophisticated engineering principles to regulate the temperature of the mound. Termites operate on very simple rules and communicate bits of incomplete, sometimes conflicting information through simple pheremone signals. No termite understands the principles of convection--they may be too simple to even grasp "it's getting hot in here," so they basically bumble around trying to respond to the signals of termites coming from above and below and feeding back their own signals to each other. But the result that emerges is very close to what a well-trained human engineer would design to accomplish the goal.

I think the "hierarchical" H in HTM may be misleading in this regard, because it's my understanding that nodes that are above are not acting in a centralizing command-and-control fashion any more than a termite who is at one level of the mound is commanding or directing one at a different level. Specifically, I don't believe that higher nodes in an HTM sort out what they think are correct or incorrect predictions coming up from below; they are only hierarchical in the sense that they operate on different levels of abstraction. If I'm wrong in this interpretation and "hierarchical" has the same meaning as in a bureaucracy, then yeah, I have some serious skepticism about Hawkins' success.

If this rivalrous process among simple, imperfectly informed agents sounds like the same kind of thing your centipede researcher is talking about, then our difference of opinion may not be so much about what can and can't work in machine intellegence, rather about what HTM is or is not.

David Beers
Pikesoft Mobile Computing
www.pikesoft.com/blog