To get straight to the point, the thing that is really interesting about the seemingly universally acknowledged fear of highly intelligent machines, is that our fear is not really of the machine like qualities, but rather their perceptibly human-like characteristics. Generally, the things we fear in machines are things like empathy, sentience, self-preservation, vengeance, and personal sovereignty. I have two theories that could explain why we have such a fundamental fear of man-like machines.
Theory 1: To allow machines to posess these qualities would put humans in a state of moral murkiness or ambiguity based on the often inhumane treatment of these machines
Evidence by how customers treat the hosts in West World. They purge their unacceptable behaviors because they believe, or at least want to believe, that hosts are non sentient which excuses those behaviors.
Theory 1b: A less forgiving addition to theory 1 is that the reason behind humans not wanting to infringe on humane rights of man-like machines because they fear that machines with superior abilities would then try to defend their humanity through violence.
ie the entire plot of Blade Runner
Theory 2: Man made machines will never be identical to humans, but will instead be so incredibly similar that it would unsettling or creepy to humans. Humans avoid "creepy" things because they are an unknown quantity; should we fear or accept? - we just don't know. The biologically ingrained reaction to the unknown is fear; think Hal's macabre voice.
Think about the robot vs human test in Blade Runner; while it is 100% accurate at predicting the classification of the examined, if the only thing separating a man and an advanced robot a flash of non-empathy at a single image, is the difference between man and machine even significant? Who knows? .. eerie.







