Artificial Emotions

audio/visual version
This article in New Scientist has me thinking about how to endow artificially intelligent machines with emotions.  This shouldn't be a hard task.

Referring to a previous post on mood, as well as one on emotions, a computerized machine can be fitted with sensors to facilitate machine learning, with unconditioned and conditioned stimuli as incentives.  Reward would be determined by the required rate intake of resources, and the net intake of resources deemed available.  Resources can include energy, and even social approval.  In the latter case, machines can detect facial features and be programmed to associate them with social signals, such as those revealed by facial expressions, for example.  The sum of all available  resources can be operationalized as mood.

The machine is then capable of attaching values to various stimuli and now emotions can be added.  Anger, for example, can be programmed as an unexpected subjective loss.  But could machines feel?

I think emotional feelings in human beings serve as signals to the working memory in the prefrontal cortex, allowing for metacognitive differentiation.  That is, with regard to our thinking about our emotional reactions, it helps to be able to distinguish each emotional response from others.  Hence, there is no reason to think such differentiation cannot be programmed into artificial intellgence systems.

The Unmeasured Self

I was watching a discussion this morning on youtube titled The Role of Psychotherapy in the Age of Neuroreceptors and Genes, and I have some thoughts.  First though, I'd like to offer a context. 

Imagine you went to your family doctor complaining of adominal pain.  Your doctor refers you to an oncologist. The oncologist has just diagnosed you with a form of cancer that has an 18% fatality rate.  It can also lead to increased risks for a host of other health problems short of death, such as cardiovascular diseases, reduced immune responding, and a general lethargy that could affect both your professional and personal lives. 

He based his diagnosis on a family history and some rather personal questions about your lifestyle and habits, including a few questions about your medical history.  He started explaining the treatment process, when you interrupt him with a question.  You ask him if he will utilize more precise diagnostics, like various scanning technologies such as MRIs.  He tells you that they are unnecessary and that he is sure enough about your diagnosis to to begin treatment. 

Treatment will involve a painless, weekly procedure that the doctor says is often effective, though different patients are more responsive to certain doctors than others.  After asking him how you'll know your getting better, he tells you that he'll chiefly base your treatment progress assessment on how you say you're feeling and whether you think it's working. 

Would you feel comfortable with this diagnostic and treatment approach?  Yet, this is exactly what psychotherapists offer their patients with major depressive disorder, which is fatal in about 18% of cases and can greatly contribute to the other maladies mentioned above, among many others.  There is often not even a paper and pencil assessment offered during diagnosis, and there is usually no attempt at objective within-patient measurements of progress during treatment.

I'm not bashing psychotherapists here and I do acknowledge that widely known research demonstrates some effectiveness for psychotherapy in the aggregate.  But still,  how is the above situation acceptable?  Is there an alternative?

I think there is.  Other than advanced brain scans, which are currently expensive, but offer much hope in the future, a behavioral economic approach can be helpful.  Since depression involves low moods, the implications of which have been described in previous posts, the resultant shift to lower risk aversion could certainly provide a definitive, quantifiable measure of progress.

This can be done in a number of ways, but to begin, you can measure the loss tolerance of patients seeking reward, such as in contests with monetary prizes.  There can be opportunites to both win or lose money, with the patient keeping any earnings.  This is an approach very widely used in psychological and microeconomic experiments.  To note, of course changes in net personal income and assets will have to be controlled for, with the gains and losses in constant proportion. 

To complement this approach, the rates of substitution between different options can be measured, to determine consistency with the more direct measure of risk attitudes, as can the  ability to delay gratification and accept sooner losses.

There are also various ways to measure relative and absolute mood levels by recording behavioral investment, revealed as energy expenditure to obtain a given amount of reinforcement(pleasure).  For example, energy expenditure can be measured with respirometers, heart rate monitors, etc.  More energy will be expended- per-unit-gained as mood increases.  This is similar to the determination in economics that marginal utility must meet marginal costs.

I personally favor the last approach, as it is more direct and requires controlling fewer variables, but I'm not sure if non-physicians are allowed to use such diagnostic techniques.  The former approaches involving money are awkward, given the treatment context.

There are still other approaches, such as measuring stress hormones in saliva or examining asymmetrical EEG data, but these are also expensive, though certainly cheaper than MRIs and other brain scans.

Of course, with all of these approaches, stable baselines must be established to measure progress against.

So, there are conceivable ways for psychotherapists to demonstrate the effectiveness of their treatments in each individual and I hope more research will go into such approaches.  The question is, will the therapists want to use them?

Duh

"Prozac and Celexa Exhibit Anti-Inflammatory Effects" is the headline in a Science Daily article today. 

This is conclusion is well within my paradigm. My paradigm says that risk aversion increases as mood decreases.  Anxiety, being an expected loss, lowers mood, which is the net rate of intake of reinforcement.  This serves the purpose of minimizing further losses of resource intake necessary for biological functioning.  The subjective value of losses increases, including those involving physical injury, as all gains and losses are translated into mood.

Given evidence, such that anxiety can worsen muscle spasms,which is well-known, and that spasms serve to protect inflamed areas of the body, to protect against further injury and/or facilitate repair, this isn't surprising.  Of course, we also know that stress can lower pain thresholds, conrolling for adrenaline spikes.

This is an example, not only of poor reporting, but of what seems to be the problem in the fields of the study of brain and behavior.  Most researchers don't seem to have an over-arching paradigm, such as the economics of behavior to guide them.

Emotions Defined

Well, behavioral economically, anyway.

What are emotions?  Emotions are reactions to changes, both past, present, and future, with the ultimate implicit goals of enhancing inclusive fitness, or at least defend adaptive homeostasis.  The latter refers to preventing a worsening of the situation for an organism capable of emotional responses, with respect to resource intake.

How many emotions are there?  Well, I personally identify eight, including anger, joy, laughter, sadness, fear/anxiety, guilt, shame, and disgust.  I'll break each of these down one at a time.

Anger involves a violation of expectations coupled with a loss.  Since expectations determine behavioral investment, any violation thereof necessarily entails a loss.  Also, obviously, it means that relevant expectations were unrealistic.  I will write more on expectations, which involves learning theory in a later post.  The exemplar experimental reference interpreted as resulting in this model can be found here.

So, the severity of an anger response is in some proportion to the subjective value of the unrewarded behavioral investment, with subjective value outlined here and here.   As well-established concept demands, anger can be "pent up", as can any reinforcing behavior in which expression is restricted.

The purpose of anger is to motivate the punishment of social wrongs, to prevent their reoccurance.  Displays of angry violence toward inanimate targets represents an evolutionary overhang, or a behavior adaptive in prehistoric environments, but not modern ones.  More specifically, there were far fewer opportunities for the inanimate to violate expectations for most of our prehistory.  The angrier one is, the more implicit memory is dominates over the explicit, at the expense of target discrimination.  Hence, violent attacks on soda machines, for example.

Another example of target indiscriminance is the typical, often externally unheard, verbal assaults on inconsiderate drivers.  This is obviously a wasteful activity even when heard by the offending driver, as in urban environments, strangers are unlikely to be encountered twice.  The benefit of adaptive punishment is lost. 

This social sensitivity with respect to strangers is due to the way we lived for most of our 200,000 or so years as a species.  There was presumably less mobility, sans modern travel techology, so strangers were probably far less frequently encountered and those met were likely to be dealt with for many years.  Punishment made sense.

To move on, joy is the flipside of anger.  It involves an unexpected gain.  Hence, the socially rewarding expression is displayed with an energy in some proportion to the newly realized "surplus" in overall intake of reward(lifted mood).  This serves the purpose of reinforcing the behavior of others that enhance inclusive fitness.  For evidence of increased motivation after unexpected reward, look here.

Now with laughter, there is merely a violation of expectations.  There is no net gain or loss associated, although the magnitude of the novelty factor can mean that a laughter response is merely the dominate among those of competing emotions. 

The presumed benefit is in the social signal that there was no gain or loss associated with violated expectations.  To illustrate in everyday terms, laughter sends an important social signal when it comes from the listener of a joke that could be taken offensively.  There is also the case in the situation in which a cared loved one unexpectedly falls down.  There is laughter in absence of perceived injury, to the inverse degree of preceived severity. There is otherwise concern, which is a signal of social value.  Similarly, when unexpected falls occur during a comedic performance, laughter ensues given the unlikelihood of injury.  Finally, cruel laughter sends the signal that the suffering of another isn't valued. 

This conception of laughter was original to me, but not within the field of behavioral research(1).  The concept may have originated with the philosopher Kant.

Now, sadness is elicited when there is a past(currently experienced), current, or future unavoidable loss.  This represents a "shortage" with respect to required intake of resources.  The expression of sadness, including crying, elicits sympathy and help from social allies to get needs met that may otherwise go unmet. 

I offer that the relative reluctance of men to cry could have to do with mate competition, but also social alliance against physical threats, and toward gainful physical cooperation.  In short, crying can be a sign of social weakness in men.  Women, on the other hand, are more likely to form mutually supportive networks with less physical competition and obviously more cooperation.  Children cty more than adults, due to the relative inability to meet their own needs, when not used as a manipulative ploy.

Fear and anxiety involve expected losses that are perceived possibly avoidable.  Possible losses often took the form of physical threats from both animal preditors and conspecifics(other people) in our extensive prehistory.  One implication is that the  fear of predation underlies sleep difficulties for the anxious, including lighter sleep and of shorter duration.  The presumed advantage is in the ability to rapidly ramp up physical arousal against night preditors.  Thus, those with sleep panic attacks often leap out of bed and land on their feed with fists clenched before they even realize what has happened.  Anxious insomniacs are often frequently occupied thinking of various ways of avoiding potential losses.

In the modern context, the levels of fear and anxiety experienced, as well as the nature of the triggers, are often inappropriate.  One can take the example of the fear of loss of face when getting anxious around srangers, in absence of any long term social consequences in most cases.

Guilt as I conceive it is simply an opportunity cost, with respect to actions taken or forgone.  It is felt when a loss to oneself, or to others that are perceived to enhance adaptive fitness is realized.  Regret is very much related.

Shame occurs in proportion to a social loss of face.  It sends the social signal that the disapproval of one's actions matters to the offending person.  It can also signal an understanding of the nature of the offense.  Shame is very much like embarrassment, but the latter does not involve a perceived loss of face, at least intitially.  It is pre-consequential evaluation.

Quickly, disgust is a response stimuli(anything that our sensse can detect) associated with either harmful germs and/or toxic substances.  The gag reflex is one of the allied reflexes that serves the purpose of ejecting harmful material in the process of consumption.  Associated differentiated facial expressions send social warning signals on the points above.  Naturally, disgust motivates the avoidance of noxious material.

Now, as a general note, as alluded to in the paragraphs on laughter, emotions can compete for simultaneous expression.  The screaming demons model very much applies here, which is general feature of competing behaviors, including thoughts.  This means that essentially, two or more emotions can be triggered at more or less the same time, with the experience and expression there of a difference between that of the greatest magnitude and that of the next greatest.  Competing emotional responses can decay at differential rates, largely depending on the frequency of repeated , but varying triggers as a result of continued evaluation and influx of new information. 

Hence, there can be net laughter, or net anger.  It is commonly observed that laughter can often follow anger.
This can happen when the associated loss fades, perhaps due to compensatory gains.  In that case, the importance of the mere novelty of the outcome becomes dominant, leading to net laughter.

Likewise, anxiety can be increasingly replaced with fear as an unexpected loss is perceived as increasingly unavoidable.  Joy can become anger, as when what one thinks is a good deal on a car intially, later is found to be relatively expensive. 

I should also mention satiation, which seems to have its own emotional experience and expressions attached, though I wouldn't personally refer to it as an emotion.  When satiated, regardless of prior expectations, there tends to be a shift toward lower physiological arousal, as evidenced in a sigh, which presumably brings breathing to a slower rhythm.

As reasoned in my posts on mood linked to in the second paragraph above, mood modulates the magnitude of emotional responses.  Lower moods augment negative emotion, and the reverse is true for positive ones.

I hope this post isn't too long, disorganized, or jargon-filled to be enjoyable.  I will perhaps offer a simpler post on this subject in the very near future.  I will also add more references to this post when time allows.


Update:  I've added some references in links above for laughter, as well as the following citation:

1. Nerhardt, G. Humor and inclinations of humor: Emotional reactions to stimuli ofdifferent divergence from a range of expectancy. Scandinavian Journal ofPsychology. 1970, 11, 185-195.

Mood, this Time with Feeling (Simpler)

I've decided to delay my first post about emotions to address continued feedback about the complexity of my original post on mood.  I acknowledge it was bad taste on my part to begin with such an entry first off.  I hope to correct that here earlier than planned..

So to jump right in, what is mood?  Mood is the net rate of intake of reinforcement, past, current, and future, discounted for time.  To break this sentence down, "reinforcment" in this case refers to anything that is pleasurable and is needed or wanted, minus the costs of obtaining it, which can include energy or effort expended, or any punishment that must be suffered in the process.   It is the pleasure that one experiences in a given moment minus any displeasure.  The displeasure can be any energy, effort, punishment or other costs for obtaining the pleasure  There is a time dimension to this, as more distant expected pleasurable experiences are valued less than those that will occur more immediately.

To dig deeper, there is a minimum amount of certain resources the body needs to function, such as energy, nutrients, water, and even social interactions and there is the amount of each available to you.
In psychological terms, the resources we need are things we don't have to learn to like, which are referred to as unconditioned reinforcers(URs).  These include the items mentioned above, along with things such as sugar or sex.  There are also conditioned reinforcers(CRs), which are simply things you can detect that predict the eventual receipt of unconditioned reinforcers.  For example, you aren't born appreciating dollar bills, but learn to like them due to the things you can purchase with them.  I just introduce these terms to readers less familiar with psychology, as they may appear in later posts.  By the way, psychologists refer to anything we can detect as "stimuli" or a "stimulus" in the singular.

As a brief note, notice that in the definition given above, I even include past intake.  This is the recognition that not only can past pleasureable or displeasurable events be remembered, but it is actually a longer term average of mood levels that, along with present and expected future intake of peasure or pain, bias current reactions to events that temporarily change mood.  For example, someone with long term depression can occasionally smile, but their longer term average mood state llimits the amount of pleasure they can experience now, while making displeasurable experiences more severe. 

Neurobiologically(skip this, unless interested), this is facilitated by the death of cells in the hippocampus in the brain, which are required to lower levels of stress hormones in the body after displeasurable events.  With higher average mood levels over enough time, these cells regenerate.  This just gives an idea of the kind of motivational inertia low average moods demand, and is very much related to depression.

Mood shapes how we value things.  The relative amount of value we place on pleasurable experiences are less when mood is lower compared to the value placed on displeasurable ones. In the absolute sense however, lower mood leads to greater values of both pleasure and displeasure, but the latter to a greater degree(change at a greater rate, with mood decreases).  Similary, higher moods lead to the opposite case. 

The basic concept is very simple.  The more of something you have, generally the less of that something you want or need.  And of course, the reverse is true.  To use a familiar example, if you are eating potato chips, the hungrier you are, the less likely you are to share chips.  But no matter how hungry you are, if you have enough chips, you will be willing to share them.  The same is true with money, of course, as first outlined by Daniel Bernoulli quite some time ago.

I will now briefly present a curve, which has the same shape as the one in the Bernoulli link above, to clarify the way mood affects how we value units of pleasure and displeasure(gains and losses):

Ignore the equation in this graph, unless interested, in which case you can learn more here and here.

You notice above the now familar term "reinforcement",a nd then and "reponse rate."  Perhaps not surprisingly, net reinforcement(again, pleasure - displeasure), should determine how much of our resources, such as time, energy, etc., we invest in a given behavior.  Hence, you see the relationship between a response rate and reinforcement rate, the former of which can also be considered simply as energy expenditure.

This sort of curve would also fit the relationship between the subjective values placed on gains and losses(pleasure and displeasure) and those on objective gains and losses, the latter if not for the influence of mood.  Replace "reinforcement rate" on the horizontal axis with "objective gains and loss values" and "response rate" with "subjective gains and loss values" on the vertical.  Now consider any point on this curve to correspond to a specific mood level. 

Whether you think about it or not, every behavior you engage in involves a prior calculation of expected gain - expected loss, in the objective sense.  A gain will move mood up on the curve and a loss, down on the curve.  So, the subjective difference between an expected gain and loss changes depending on the mood level.  Moving up on the curve means that both gains and losses are valued less, and the difference is greater in favor of net gains, compared to objective gains and losses.  Similarly, moving down on the curve means that both subjective gains and losses grow in value, but those of losses more quickly.  .

So, mood is like a general currency in which all pleasure and displeasure is converted. You can refer to the net units of mood, or pleasure - displeasure, .as units of reinforcement, or use the terms economists use, such as utils or even units of demand. You can even just use "units of pleasure". It's not really imporant, unless trying to discuss these concepts with people in related fields.

To sum up, this is very similar to the income effect, in which the higher the monetary income, the less you value each additional dollar earned, all else being equal.  For example, see the curve below:

Utility is the same as net pleasure for our purposes, or you can say "net motivation." 

This final illustration makes it easy to consider some effects of changes in mood.  When mood decreases, there is a shift in preference toward more "profitable" behaviors.  These include the things that bring us the most pleasure, quickest.  Good examples are sex, recreational drugs, and high calorie foods containing fat and sugar.  It is perhaps anecdotally clear that when we feel down, we are more susceptible to sexual seduction, drinking alcohol, or eating comfort foods.  This is just as a company that becomes less profitable begins to shift resources to more profitable operations, cutting or eliminating less profitable ones.  There is generally a need for more income, or a higher rate of intake of resources.  You may also realize that this explains why people who are depressed also have more trouble delaying gratification with implications for procrastination, among other difficulties.  Perhaps it is also clear why negative emotional responses are more severe when mood is low.
 
Speaking of depression, the above now gives us a good model.  Depression is a motivational trap in which as mood decreases, again, the subjective values of gains and losses both increase, but those of losses more quickly.  Hence, the seeming confusion resulting when there is a greater desire or need to engage in certain behaviors, but less motivation to do so.  This is seen particularly in many depressed patients, especially women, who may not seek sex as much when depressed, but may be more likely to engage in it when offered with relatively less effort required.  On the other hand, some become more aggressive when seeking sex, if the costs of doing so are sufficiently low.  From a prescriptive point of view, this supports the need for psychotropic drugs to help pull patients out of this often self-feeding motivational trap.  This should increase the ability of patients to then invest their time and other resources (including attention) in counseling for greater benefit.

Finally, why do we have mood and why does it work this way?  Well, these mood-related effects help motivate us to conserve more of our resources the lower the availability of them becomes.  Though it may seem maladaptive for the brain to work this way, and it is in the modern environment, in an environment of relative scarcity it makes a lot of sense.  Such environments are those we inhabited within the stereotypical hunter-gatherer groups for our 200,000 year prehistory. 

I hope I avoided getting too technical here, while offering enough information to explain this conception of mood.  I will appreciate any feedback on my success in his sense.

What is Mood? A Behavioral Economic Definition

This first post may seem at least a bit technical, despite the simplifications.  I just want to offer some of the more precise foundations of the approach this blog will discuss.  Simpler, more concise posts on mood will follow.  I start with an operationalized definition of mood, which I represent as what we really mean when we talk about higher or lower states of affect.

Mood simply represents the sum of the perceived available, temporally discounted net reinforcing options, or the baseline rate of intake of reinforcement, in the context of a required net inake. For example, at a given level of energy expenditure, there is a required net caloric intake. The net intake of this specific category of reinforcer (energy) is translated into mood, which is the ultimate currency in the brain.

Around 80% of people are risk/loss averse, meaning they value losses more heavily than gains. This is increasingly true as mood decreases, because there is the equivalent of the wealth effect operating on the brain. Hence, as mood decreases, net reinforcement shifts toward more immediate, more "profitable" gains, as resource conservation occurs. This is manifest as greater tendencies toward promiscuous behavior, preference shifts toward higher calorie foods (unless body image supresses it), and drug abuse. Also, because the value of losses loom ever greater than that of gains, negative emotional responses are augmented while positive ones are diminished.

This can be summed up nicely here with a greatly simplified, but relevantly complete mathematical expression of net motivation. U = N [ g / ( h + g ) - L / ( h - L )], where U = net utility or net motivation (subjective net expected gain or loss, or you can call this simply demand), g = objective gain, L = objective loss or response cost, h = baseline rate of intake of reinforcement, or "have" (mood), and N = required rate of reinforcement (Need). Ultimately, this is a model relating the required rates of intake of unconditioned reinforcers to the resources perceived available, with the ultimate goal being to optimize inclusive fitness, or reproductive success.  This is a reinterpretation of the various matching law models.

This would explain why, for example, girls with relatively higher stress levels actually enter puberty earlier than girls with lower stress. The brain makes an economic calculation based on environmental input indicating an environment relatively hostile to reproductive success, and hence a higher quantity reproductive strategy is selected for. Girls who have higher stress and enter puberty earlier have more sexual partners on average and have sex earlier than those with lower stress levels.

This equation models depression as well, and clearly defines the crux of the problem. Those who are depressed, perhaps seemingly paradoxically, value gains more heavily, but net motivation decreases because the value placed on losses increases at a greater rate as mood decreases. This is consistent with research, demonstrating, for example, that participants diagnosed with depression are willing to expend less energy for reinforcement, but consume more than the non-depressed when response costs are reduced.

Increases in net motivation allow for greater behavioral investment in seeking natural reinforcement, which can lead to habits, with proper psycho-education, that can maintain higher mood levels sans medication. This should be the ulitmate goal of any plan to treat non-purely neurological depression.

This model also predicts that anti-depressants will offer little beneft for the dysthymic, as the curve for net subjective value (utility) flattens as mood increases. This prediction is consistent with experiment and other data.  The model also suggests explicit roles for dopamine in mood and motivation, as it represents the subjective magnitude of gains and losses. Hence, the model predicts the inverted U-shaped curve for the effects of stimulants on net motivation, and the effects of stimulants on attention and negative emotional behavior.

This is a self- and externally consistent model of mood, emotions, and motivation. Hopefully, this post can allow one to begin to see where this model comes from. Those wanting more references, or who have any questions about how the dots connect, don't hesitate to ask. This is not extremely complex, but it is a different way of thinking about mood, motivation, thinking, and behavior for many.