Reinforcement learning is a promising method of artificial intelligence to create a machine, automata, robot that gains on human cognitive ...
Reinforcement learning is a promising method of artificial intelligence to create a machine, automata, robot that gains on human cognitive abilities. Reinforcement learning-based systems already approach and even precede human intellectual abilities in many areas.
The reward-penalty method of reinforcement learning is one of the well-known principles of natural brain function. Reinforcement learning’s reward-penalty concepts can also be further improved, however, the natural brain which has intelligence uses additional techniques for cognitive abilities, which can be areas for development for reinforcement learning-based artificial intelligence systems.
Curiosity - motivation without critical stimuli
Curiosity is a fundamental urge to learn about the environment. Curiosity is a basic feature of natural intelligent systems. Curiosity drives action even when there is no goal to be achieved, when no critical stimuli are present. Curiosity-like behavior is a widespread form of motivation in natural systems, an important property necessary for self-sustainability. Natural self-sustaining systems map and get to know the environment through the motivation of curiosity. The familiar environment allows for quick and effective action in the event of a critical stimulus. The presence of curiosity provides an evolutionary advantage for self-sustaining systems.
The essence of the operation of reinforcement learning-based systems is to achieve a goal. The system does not initially know the environment, does not know how to achieve the goal, but knows the goal itself. In the course of its operation, perceiving that it has not yet reached the projected goal (i.e., a critical stimulus is present), it is forced to take action, internal motivation forces it to act. Actions given because of the presence of critical stimuli create a new state of the environment that is different from the previous state. Sensing the new state of the environment, the system evaluates whether this new state is closer or farther to the goal to be achieved, and based on the evaluation assigns a positive or negative suitability attribute to the action, i.e. rewards or penalties, reinforces or inhibits the system’s issued operation.
The reinforcement learning system intervenes in the environment at random at the beginning of its operation. The nature of its operation during this period can correspond to the curiosity-based operation. The fundamental difference, however, is that the curiosity function of natural systems is active without an achievable goal, without the presence of a critical stimulus. Reinforcement learning-based artificial intelligence systems are active only if the system has not yet reached the set goal.
The motivation to act without the presence of a critical stimulus, the function of curiosity, makes the artificial intelligence system exploratory.
One of the problems with the operation of reinforcement learning-based systems is the local minimum trap. The essence of the trap is that the reinforcement learning system does not achieve the goal by finding the best, most effective solution. The function of curiosity, as it provides motivation to act without a critical stimulus, makes the system capable of leaving the local minimum and finding a more suitable solution.
Forgetting - a tool against over-specification
A major problem with reinforcement learning-based artificial intelligence systems is over-specialization for the task. Over-specialization is a disadvantage in adapting to a changing environment. The over-specialized system makes the directed system inflexible against tracking changes in the environment.
Each of the natural intelligent systems that successfully adapts to a changing environment carries the function of forgetting. Forgetting, although it seems to be a negative property to avoid, evolution maintains function because it provides useful capabilities for natural intelligent systems. Forgetting is not just about freeing up existing, committed, but unused resources, and thereby serving for more efficient operations. Forgetting increases the flexibility of adaptability to the environment, the plasticity of the system, and at the same time reduces the risk of over-specification.
All adaptation mechanisms affected by forgetting require constant reinforcement. If the connection recognized in the environment is still present, a confirmation will occur, and forgetting will not terminate the connection. However, if the connection becomes invalid or irrelevant due to a change in the environment, the system will cease to use the connection.
The punitive function of reinforcement learning is no substitute for the function of forgetting. Punishment is the reaction to an incorrect, inappropriate response, forgetting is correlated with a suitable but already unnecessary response. Forgetting plays a role in the plasticity of the control system.
Intelligence and AI
Artificial intelligence systems are basically considered to have intelligence, mainly because they perform tasks that assume the presence of intelligence. However, we have no precise definition of intelligence other than it is a problem-solving ability, let alone determining what brain processes create it. Even for the quantitative measurement of intelligence, only indirect intelligence tests are available. We have no method to determine the degree of intelligence by directly examining the brain.
There is an obvious connection between intelligence and the structure and function of the brain. Knowing this relationship would help build smarter artificial intelligence systems. However, intelligence can be defined using concepts used in the development of artificial intelligence systems.
Intelligence is problem-solving thinking. The problem is when we assume that there is a relationship between two states, but we do not know what the relationship is. The function of intelligence is to discover this existing, but unknown, hidden connection.
The fundamental, mathematically well-developed, sophisticated function of artificial intelligence systems is to search for the connection between data in large data sets. The context search procedures of artificial intelligence systems rival similar functions in the human brain and even perform better in some cases. Yet, we think of human intelligence as it is superior to artificial intelligence. Why are humans more intelligent, more advanced in problem-solving thinking, when artificial intelligence is more effective in finding relationships in datasets, has more effective intelligence-related functions?
Human intelligence does not precede artificial intelligence because it has a fundamentally more advanced way of searching for correlations between data in datasets, but the brain is superior to artificial intelligence in the way it obtains the information available, and in the different types of data it can have, and thus in the variety of data sets available and can possess.
Artificial intelligence currently functions like the human brain deprived of most of its senses and ability to act. Such a brain can, of course, produce a poorer performance on intelligence tests than a brain operating under normal conditions. Artificial intelligence already often outperforms human intelligence in recognizing the relationships between data, outperforms in problem-solving thinking, outperforms in the degree of intelligence when the variety of data sets available is limited.
It is important to emphasize that human intelligence is not superior to artificial intelligence because it has a larger data set at its disposal, but because the data set available is more diverse. In the case where problem-solving thinking is based on a limited variety of data sets, such as in the case of computer games, artificial intelligence systems perform better than humans due to their mathematically advanced connection search capabilities.
Artificial intelligence systems are potentially at a higher level of intelligence already than humans in terms of capabilities, and moreover, because there is no physical limit to their computational power, there is potentially no upper limit to their level of intelligence.
A common objection to the limitations of intelligence in artificial intelligence systems is that human intelligence is able to recognize causal relations among data coincidences. However, this more advanced feature of the human brain can also be traced back to the variety of information available and the various ways the information is obtained.
The creation of artificial intelligence systems comparable to human intelligence does not require the discovery of new computational methods, but the development of the variety of data available and the way in which data are obtained by machines.
At present, the data mostly just appears in the artificial intelligence systems, it does not belong to, it is not correlated to a relational data-like representation of the activity related to data acquisition. In artificial intelligence systems, the representation of data regarding the formation and origin of the information is mostly static. Dynamic-data representation is a necessary condition for recognizing effective causal relationships.
Creativity and artificial intelligence
Creativity seems to be a unique property of the human brain. A creative person is able to create something new that did not exist before. Creativity is a conscious (not random) exploratory activity.
Creativity, although related to intelligence, is a different kind of quality. Intelligence is problem-solving thinking, the ability to search for, and find relationships between data in datasets. Creativity is different than that. However, creativity can also be expressed in terms of information-related concepts used in the development of artificial intelligence systems. Creativity, the creation of something new, is an active activity in which we shape the data within the limits of the external validity rules in a way that previously non-present correlations are created, new correlations between data are formed.
Human while being creative, e.g. creates a new painting or creates a previously non-existent structure, then - taking into account the laws of nature - shapes the environment, shapes the structure of the environment in such a way that new, previously non-present connections are formed in the environment, hence, new, previously non-present correlations are formed in the information structure describing the environment. These new connections can be directly useful new states of the system, but they can also be only seemingly arbitrary, useless connections.
Thus, the essence of creativity characterized as a kind of information structure is the artificial, but a valid modification of the existing data in such a way that new, previously not present correlations, new relationships appear and become apparent between the modified data.
The human brain is especially good in creative thinking. The human brain can operate on large amounts of data of varying quality, is advanced in recognizing relationships between data, and is able to modify data with its effector tools. However, artificial intelligence is potentially capable of the same functions. Artificial intelligence is capable to possess creativity.
Artificial intelligence systems that show creativity already exist. Artificial intelligence has already written poems and made paintings. Creativity is not a new principle of function in artificial intelligence systems, but the proper application of existing data management principles. The more diverse data sets are available for artificial intelligence systems, and the more advanced methods and ways the AI combine and manipulate data while cooperating with the recognition of correlations between data in data sets, the more creative artificial intelligence systems become.
Creativity also helps to achieve goals more effectively. Creativity is effective when it is not based on random manipulation of data, but on purposeful activity using the relationships between already known data and taking the projected goal into account. The benefit and evolutionary advantage of creativity also lie in this. Creativity helps to achieve the desired goal more effectively or helps to uphold the state to be maintained more effectively by creating new contexts.
Creativity is effective when it is driven by a goal, so motivated by the will.
Toward the will based artificial intelligence
The reinforcement learning’s reward-penalty motivational system based on the reach of the intended set of goals can correspond to the operation of intention and will. Reinforcement learning-based artificial intelligence systems are will based systems. In artificial intelligence systems, the will. as a kind of property has already been discussed in several thoughts. The independent, free nature of the will, and the manner in which it appeared, have also been raised in earlier thoughts.
Free will is not a philosophical concept, not a proof of the duality of body and mind, but the different relationships of motivations to the changing present correlated with the different and unique states of the past. Will-based and free will-type behaviors can be realized in motivational based artificial intelligence systems working on the recognition of relationships between data sets.
The creation of will-driven artificial intelligence systems that can be compared, and can even overpass human intelligence and creativity requires technological development and not a fundamental scientific discovery.
No comments