Sight

(This is a statement of some research I was prevented from doing at the University of North Dakota.)

It is said that a person looks with their eyes, but sees with their brain. I have given some thought to exactly what this means.

The reason is simple. With a computer is it easy to do arithmetic. That is, add an subtract. A computer can outdo any person in this, no questions asked. Yet, a person looks at an image, and can immediately identify what the image is a picture of. For a computer, this is much more difficult.

I have noticed that some modern computers have been able to do this. Usually, attempts I've seen accomplish this by a brute force method. That is, the processing power on the computer is simply increased until it is able to do the slow identification methods in a reasonable time.

Yet, this is not comparable with the way people do it. The human mind works by sending signals down neurons, which travel at roughly the speed of sound. Yet, computers send comparable signals down wires at roughly the speed of light. The speed of sound is something like 1/8 mile per second. The speed of light is something like 186,000 mile per second, which is more than a million times faster.

This is why computers do arithmetic so much quicker than humans.

This means that computers, if comparable algorithms are used, should be able to recognize a picture about a million times faster than a human.

To be honest, it is known that humans use parallel processing a lot more than computers. That is, a computer will typically do one thing at a time, but will do it very fast. (Ignoring such things as pipelining, and parallel i/o, which are commonly done.) This would give humans an edge. Of course, that means that the algorithm used must depend very heavily on parallel processing.

I have heard it said that Greek philosophers used to say that when a person looks at a chair, they don't see the actual chair, but an idealized image of the chair. I remember hearing this in high school, but have not read enough Greek philosophy to verify it.

Be that as it may, I would propose that people see exactly like that. That is, a person doesn't "see" a chair, but only an idealized image of a chair.

Therefore, a person has, in their head, a "model" of reality. This is what is actually seen. The senses, then, verify this model on a regular basis. The model is altered as needed to accommodate the senses.

This is why razzle-dazzle works. That is, during the first world war, the German submarine took their tole on shipping. Therefore, the allies just colored the ships with bright colors in wild patterns. The human eye of the submarine captain couldn't see the ships. That is, he knew they were there because he could see the color, but could not make out the ships.

What was happening was that the model of reality the submarine captain included idealized ships, but the information he was getting from his senses didn't verify that fact, so his model did not reflect reality, and he acted accordingly. The torpedoes launched were aimed, not at the ships that existed, but at the ships which were in the captains model, which was in error.

Another manifestation of this is the blind spot. Every human eye has a blind spot, which is the point were the optic nerve enters the eye. Yet people are not aware of this until it is show by specifically placing something in the blind spot. The reason people don't see it is because the logic which interprets the sensual data is aware of that spot, and verifies the model accordingly. Since no data is available for that area, the eye simply extrapolates the model.

Optical illusions are another example of the same thing. That is, the eye is presented with information which causes the mind to update its model wrong. Therefore, a horizontal line and a vertical line of the same length look as if the vertical line is longer. That is because the model probably exaggerates the vertical, since humans don't have much vertical control over reality. Similarly, a trapezoid is distorted to look like a rectangle because in the civilized word there are so many rectangles that the human mind had taken to expecting that when it modifies the model. I've been told (and find it easy to believe) that a person who grew up in an uncivilized forest would not be fooled by that type of optical illusion.

Another evidence of this method of processing can be seen when a person is suddenly put into a situation which is different from they have seen. There is a bit of hesitancy, lasting for seconds at times, when the person cannot identify their surroundings. This is exactly equivalent to loading the pipeline on a pipelined computer. That is, the old model is no longer valid to the point where a new model needs to be created. If the new surroundings are something the person has seen before, then the model is easily recalled from memory. If the surroundings are made from objects the person is familiar with, then, although it will take longer, the model will be made reasonably easily.

If the new surroundings are made up of things the person is totally unfamiliar with, then it will take the person much longer to identify the surroundings. In fact, it can takes minutes for the person simply to get every piece of data entered into the model - the person needs to create new entries in the list of model objects, as well as rearrange the existing objects. This takes time.

Nor is this model fixed. That is, I recall an experiment I saw on television once where a football player was given a set of glasses that inverted all the images before they entered the eye. (This would mean that the images on the retina were, in fact, not up side down, but right side up.) He was shown trying to throw a football early on, but it didn't go right because he was throwing to a position in his model, which was wrong. After a while (a week, I think) of wearing the glasses, he was shown throwing the ball again. This time, he threw as good as he ever did. The reason, of course, is that the logic verifying his model was now acclimated to the difference in the way it needed to be verified.

A similar experience can be seen by a person who is made to see after being blind since birth. This can happen if a person is born with cataracts, and the cataracts are removed at a later date. In this case, sight has never been used to update the model before, so the person's model of reality doesn't include light. Therefore, even though the person's eyesight is normal, they might walk into walls and doors exactly like a person who was blind. This will continue until all the models the person has are updated to include light. I'm not sure how long that takes.

In high school we did another experiment. I think it may have been in Chemistry class, but did not include chemistry. In this experiment, everyone was given a black box, actually, a shoe box colored black. Inside, an object had been placed. The object of the experiment was to determine what the object was without opening the box. That is, the model had to be generated from very limited data, a set of data we were not used to working with.

It was hard, and I don't know that anyone figured it out properly, so once we were allowed to open the box we were surprised.

For me, all this was to be used to generate a seeing robot. That is, I could copy the natural algorithm, then I could generate a computer that could see as easily as a person. I could look at how to arrange micro processors so they would be best used in the image decoding process. (I think my initial designed used four processors, slow processors by todays standards, and I probably would have opted for slow cheap processors rather than expensive fast ones. I like a challenge.)

(Also, for the people who are not familiar with the way computers are programmed, it should be pointed out that using lessons learned in natural systems, seeing in this case, is not unusual in Computer Science. A common approach to a problem is to try to find a similar natural system, then emulate that system as best you can. Nature is no fool, and has solved many problems that Computer Scientist are just now figuring out.)