Evolved foraging behaviors

2 collaborators

Eden Forbes (Author)

Gabriel Severino (Author)

WHAT IS IT?

This is a brain-body-environment (BBE) model in which a single turtle must use various kinds of perceptual signals to approach a target in the environment. Instead of prescribing rules that define the turtles' behavior, here behavior is evolved using NetLogo's BehaviorSearch. As such, many different kinds of behavior may be observed that solve the same sensorimotor foraging task. The diversity and robustness of those behaviors is of principle interest with this kind of model.

The BBE model is defined by:

Brain: a continuous-time recurrent neural network (CTRNN). This simple neural network couples perceptual inputs from the environment to motor outputs by the turtle. The parameters for the network are subject to an evolutionary algorithm. Random parameter sets are largely ineffective, but with stochastic search can readily converge on unique solutions to the foraging task.

For an interactive CTRNN model and more information, users can see the CTRNN NetLogo model (imported here as an .nls script).

Body: a turtle! The single turtle occupies a discrete position ([x, y] coordinate) which is updated according to the outputs of two CTRNN nodes with some loss from friction. The body also determines how the turtle perceives environmental information. There are three possible options.

Visual: The turtle perceives the Euclidean distance between itself and the target directly. That distance information is used as an input to each node of the CTRNN governing its behavior.

Chemical: The target emits a uniform chemical gradient. The turtle perceives the gradient value at the patch it currently inhabits. That gradient information is used as an input to each node of the CTRNN.

Tactile: Either using distance OR gradient information, the turtle perceives information only if it is nearly at rest, assuming that in motion it is unable to perceive environmental stimuli due to perceptual noise from its own movement.

Environment: a single target, which in chemotaxis tasks emits a gradient with some prescribed steepness. The environment is wrap-around (a torus) such that the turtle can freely move in any direction.

HOW IT WORKS

Each timestep, the turtle perceives environmental information. Then, the CTRNN is integrated with that environmental information as input. The resulting outputs are used to determine the motion of the turtle, whose location in the environment is updated accordingly. This process repeats until the duration of the simulation is up.

HOW TO USE IT

There are two primary ways of using this model: evolving behaviors and analyzing behaviors.

Evolving behaviors

Evolving behaviors requires downloading and installing NetLogo's BehaviorSpace module. BehaviorSpace includes several stochastic search algorithms that can be used to find solutions for the various foraging tasks described here. We have attached BehaviorSpace files tuned to the various perceptual situations described above that users can use right away. Some important things to check when evolving behaviors:

1) Make sure BehaviorSpace is using setup-evo rather than setup (the former prepares the simulation for an evolutionary algorithm, while the latter is tailored to load a particular CTRNN parameter set).

2) Make sure that you are using the perceptual situation you intend.

The 'gradient?' switch determines whether the turtle receives distance information or environmental gradient information

The 'tactile?' switch determines whether the turtle receives environmental information constantly or only if it is moving below a certain velocity.

BehaviorSpace will automatically show you how the best fitness performance changes over the course of evolution. The parameter set(s) that yield the best solution(s) are also automatically saved in a .csv file.

Analyzing behaviors

To pick a solution to analyze, specify the filepath in 'genome-location' and, if there are multiple solutions in that file, select the individual solution using 'focal-index'. Then, make sure the 'load-in-NS?' is ON (OFF will load in a random CTRNN parameter set, which can be interesting as a comparison but which will almost never yield effective behaviors).

As when evolving behaviors, make sure the 'gradient?' and 'tactile?' are set to the correct perceptual situation. An evolved solution that works in one perceptual tasks likely will not work in a different perceptual task (although some may transfer!)

'setup' will then configure the simulation for you. You will see the CTRNN parameters in the various cells below the viewing window.

'go' will run the simulation for 'runtime' at integration stepsize 'stepsize'. The CTRNN inputs and outputs are plotted over time to the right of the viewing window. Further to the right, the two CTRNN nodes that determine motor behavior are plotted together.

THINGS TO NOTICE

Try to determine how changes in the perceptual inputs correspond to changes in motor behavior. Do this both via 'ethology' (look at the viewing window and see how the turtle changes its trajectory depending on its orientation to and distance from the target) and 'neuroethology (look at how perceptual inputs change the behavior of motor outputs in the plots).

THINGS TO TRY

For any given solution, play with the parameters of the CTRNN and see how they change the resulting foraging behavior. At what point does the behavior break down? Can you make the behavior even more efficient?

In chemical situations, you can also change the 'steepness' of the gradient. How well do solutions to the chemotaxis task work when the gradient changes?

EXTENDING THE MODEL

Other perceptual circumstances/rules can be added however the user would like to relate behavior to perception or via the construction of new kinds of environmental objects.

Multiple targets could be added as well, either being consumed by the turtle or competing for the turtle's attention.

This version of the task also has no obstacles. Adding environmental heterogeneity would change the task significantly.

NETLOGO FEATURES

The model heavily relies on BehaviorSearch. The CTRNN class operates entirely without turtles, jsut taking advantage of NetLogo as a platform for numerical integration.

RELATED MODELS

The CTRNN class is necessary for this model to function.

CREDITS AND REFERENCES

The evolution, analysis, and comparison of various BBE models with 'visual', 'chemical', and 'tactile' foraging/taxis strategies emulated in this model have been published in multiple manuscripts (based on C++ implementations of these models):

Agmon, E., & Beer, R. D. (2014). The evolution and analysis of action switching in embodied agents. Adaptive Behavior, 22(1), 3-20. https://doi.org/10.1177/1059712313511649

Forbes, E. J., Todd, P. M., & Beer, R. D. (2025). The role of signal in saltatory pursuit of cryptic stationary targets. Movement Ecology, 13(1), 89. https://doi.org/10.1186/s40462-025-00611-z

Izquierdo, E. J., & Lockery, S. R. (2010). Evolution and analysis of minimal neural circuits for klinotaxis in Caenorhabditis elegans. Journal of Neuroscience, 30(39), 12908-12917. https://doi.org/10.1523/JNEUROSCI.2606-10.2010

Severino, G. J., & Barwich, A. S. (2024). Evolution and analysis of respiratory odor navigation in embodied agents. In Artificial Life Conference Proceedings 36 (Vol. 2024, No. 1, p. 33). MIT Press. https://doi.org/10.1162/isal_a_00751

Comments and Questions

Please start the discussion about this model! (You'll first need to log in.)

Click to Run Model

extensions [csv]
__includes ["CTRNN.nls"]

turtles-own [thrust torque velocity theta sweights dtrav initdist fitness]
patches-own [resource-density]

globals [
  max-angle max-thrust transient max-patch
]

to make-gradient
  let source-patch one-of patches
  ask patches [
    set resource-density exp (-((steepness * distance source-patch) ^ 2))
  ]
  let max-resource max [resource-density] of patches
  ask patches [
    set pcolor scale-color green resource-density 0 max-resource
  ]
  ask source-patch [set pcolor red] ;; Highlight the source patch
end 

to setup
  ca
  reset-ticks
  make-gradient
  
  set max-angle 20 ; RADIANS
  set max-thrust 2
  set transient 1000
  set max-patch max-one-of patches [resource-density]
  
  create-turtles 1
    [
      let init-pos one-of patches with [resource-density < 0.05]
      set size 8
      set color pink
      set xcor [pxcor] of init-pos
      set ycor [pycor] of init-pos
      set heading random 360
      set initdist distance max-one-of patches [resource-density]
      pen-down
    ]
  
  ctrnn-initialize 3
  
  ifelse load-in-NS?
  [;; LOAD IN PARAMETERS ;;
    let best-genotype csv:from-file genome-location
    
    set sw1 item 17 (item focal-index best-genotype)
    set sw2 item 18 (item focal-index best-genotype)
    set sw3 item 19 (item focal-index best-genotype)
    ask turtles[
      set sweights (list item 17 (item focal-index best-genotype) item 18 (item focal-index best-genotype) item 19 (item focal-index best-genotype))
    ]
    ctrnn-set-gains 0 1
    ctrnn-set-gains 1 1
    ctrnn-set-gains 2 1
    
    set tau1 item 2 (item focal-index best-genotype)
    ctrnn-set-neuron-tau 0 item 2 (item focal-index best-genotype)
    set tau2 item 3 (item focal-index best-genotype)
    ctrnn-set-neuron-tau 1 item 3 (item focal-index best-genotype)
    set tau3 item 4 (item focal-index best-genotype)
    ctrnn-set-neuron-tau 2 item 4 (item focal-index best-genotype)
    
    set bias1 item 5 (item focal-index best-genotype)
    ctrnn-set-neuron-bias 0 item 5 (item focal-index best-genotype)
    set bias2 item 6 (item focal-index best-genotype)
    ctrnn-set-neuron-bias 1 item 6 (item focal-index best-genotype)
    set bias3 item 7 (item focal-index best-genotype)
    ctrnn-set-neuron-bias 2 item 7 (item focal-index best-genotype)
    
    set w11 item 8 (item focal-index best-genotype)
    ctrnn-set-connection-weight 0 0 w11
    set w12 item 9 (item focal-index best-genotype)
    ctrnn-set-connection-weight 0 1 w12
    set w13 item 10 (item focal-index best-genotype)
    ctrnn-set-connection-weight 0 2 w13
    set w21 item 11 (item focal-index best-genotype)
    ctrnn-set-connection-weight 1 0 w21
    set w22 item 12 (item focal-index best-genotype)
    ctrnn-set-connection-weight 1 1 w22
    set w23 item 13 (item focal-index best-genotype)
    ctrnn-set-connection-weight 1 2 w23
    set w31 item 14 (item focal-index best-genotype)
    ctrnn-set-connection-weight 2 0 w31
    set w32 item 15 (item focal-index best-genotype)
    ctrnn-set-connection-weight 2 1 w32
    set w33 item 16 (item focal-index best-genotype)
    ctrnn-set-connection-weight 2 2 w33
  ]
  [;; RANDOM PARAMETERS ;;
    randomize-ctrnn-parameters
    
    set tau1 ctrnn-neuron-tau 0
    set tau2 ctrnn-neuron-tau 1
    set tau3 ctrnn-neuron-tau 2
    
    set bias1 ctrnn-neuron-bias 0
    set bias2 ctrnn-neuron-bias 1
    set bias3 ctrnn-neuron-bias 2
    
    set w11 ctrnn-connection-weight 0 0
    set w12 ctrnn-connection-weight 0 1
    set w13 ctrnn-connection-weight 0 2
    set w21 ctrnn-connection-weight 1 0
    set w22 ctrnn-connection-weight 1 1
    set w23 ctrnn-connection-weight 1 2
    set w31 ctrnn-connection-weight 2 0
    set w32 ctrnn-connection-weight 2 1
    set w33 ctrnn-connection-weight 2 2
    
    ctrnn-set-gains 0 1
    ctrnn-set-gains 1 1
    ctrnn-set-gains 2 1
    ask turtles[
      set sweights (list ((random-float 40) - 20) ((random-float 40) - 20) ((random-float 40) - 20))
    ]
    set sw1 item 0 [sweights] of one-of turtles
    set sw2 item 1 [sweights] of one-of turtles
    set sw3 item 2 [sweights] of one-of turtles
  ]
end 

to setup-evo
  ca
  reset-ticks
  make-gradient
  
  set max-angle 20
  set max-thrust 2
  set transient 1000
  set max-patch max-one-of patches [resource-density]
  
  create-turtles 1
    [
      let init-pos one-of patches with [resource-density < 0.05]
      set size 8
      set color pink
      set xcor [pxcor] of init-pos
      set ycor [pycor] of init-pos
      set heading random 360
      set sweights (list sw1 sw2 sw3)
      set initdist distance max-one-of patches [resource-density]
    ]
  
  ctrnn-initialize 3
  ctrnn-set-gains 0 1
  ctrnn-set-gains 1 1
  ctrnn-set-gains 2 1
  
  ctrnn-set-neuron-tau 0 tau1
  ctrnn-set-neuron-tau 1 tau2
  ctrnn-set-neuron-tau 2 tau3
  
  ctrnn-set-neuron-bias 0 bias1
  ctrnn-set-neuron-bias 1 bias2
  ctrnn-set-neuron-bias 2 bias3
  
  ctrnn-set-connection-weight 0 0 w11
  ctrnn-set-connection-weight 0 1 w12
  ctrnn-set-connection-weight 0 2 w13
  ctrnn-set-connection-weight 1 0 w21
  ctrnn-set-connection-weight 1 1 w22
  ctrnn-set-connection-weight 1 2 w23
  ctrnn-set-connection-weight 2 0 w31
  ctrnn-set-connection-weight 2 1 w32
  ctrnn-set-connection-weight 2 2 w33
end 

to qsa
  ask turtles[
    let ips map [i -> i * resource-density] sweights
    ctrnn-set-external-inputs 0 item 0 ips
    ctrnn-set-external-inputs 1 item 1 ips
    ctrnn-set-external-inputs 2 item 2 ips
    ctrnn-euler-step stepsize
  ]
  if ticks >= runtime [
      stop
    ]
  tick
end 

to go
  if ticks < transient [
    ctrnn-euler-step stepsize
  ]
  
  if ticks > transient [
    ask turtles[
      
      ifelse gradient?
      [
        ifelse tactile?
        ;; Tactile perception: gradient ;;
        [ifelse thrust < 0.1
          [let ips map [i -> i * resource-density] sweights
            ctrnn-set-external-inputs 0 item 0 ips
            ctrnn-set-external-inputs 1 item 1 ips
            ctrnn-set-external-inputs 2 item 2 ips]
          [ctrnn-set-external-inputs 0 0
            ctrnn-set-external-inputs 1 0
            ctrnn-set-external-inputs 2 0]
        ]
        ;; Chemical perception: gradient ;;
        [let ips map [i -> i * resource-density] sweights
          ctrnn-set-external-inputs 0 item 0 ips
          ctrnn-set-external-inputs 1 item 1 ips
          ctrnn-set-external-inputs 2 item 2 ips]
      ]
      [
        ifelse tactile?
        ;; Tactile perception: distance ;;
        [ifelse thrust < 0.1
          [let ips map [i -> i * (1.0 / (distance max-patch))] sweights
            ctrnn-set-external-inputs 0 item 0 ips
            ctrnn-set-external-inputs 1 item 1 ips
            ctrnn-set-external-inputs 2 item 2 ips]
          [ctrnn-set-external-inputs 0 0
            ctrnn-set-external-inputs 1 0
            ctrnn-set-external-inputs 2 0]
        ]
        ;; Visual perception: distance ;;
        [let ips map [i -> i * (1.0 / (distance max-patch))] sweights
          ctrnn-set-external-inputs 0 item 0 ips
          ctrnn-set-external-inputs 1 item 1 ips
          ctrnn-set-external-inputs 2 item 2 ips]
      ]
    ]
    
    ctrnn-euler-step stepsize
    
    ask turtles[
      set thrust (ctrnn-neuron-output 0 + ctrnn-neuron-output 1) * max-thrust
      set torque (ctrnn-neuron-output 0 - ctrnn-neuron-output 1) * max-angle
      set velocity (velocity * 0.9 + stepsize * thrust)
      set dtrav (dtrav + stepsize * velocity)
      set theta (stepsize * torque)
      set xcor (xcor + (stepsize * velocity * sin(heading + theta)))
      set ycor (ycor + (stepsize * velocity * cos(heading + theta)))
      set heading (heading + theta)
      if heading > 360
      [set heading (heading - 360)]
      if heading < 0
      [set heading (heading + 360)]
    ]
    
    
    if ticks >= runtime [
      ask turtles[
        set fitness (1 - (distance max-patch / initdist))
        if fitness < 0
          [set fitness 0]
;        show word "Final position resource density: " resource-density
;        show word "Distance traveled: " dtrav
;        show word "Initial distance: " initdist
;        show word "Final distance :" distance max-one-of patches [resource-density]
        show word "Fitness: " fitness
      ]
      stop
    ]
  ]
  tick
end

There is only one version of this model, created about 8 hours ago by Eden Forbes.

Attached files

File	Type	Description	Last updated
Chemotaxis.bsearch	extension	BehaviorSearch file for Chemotaxis	about 5 hours ago, by Eden Forbes	Download
CTRNN.nls	background	Required CTRNN .nls file	about 5 hours ago, by Eden Forbes	Download
Evolved foraging behaviors.png	preview	Preview for 'Evolved foraging behaviors'	about 8 hours ago, by Eden Forbes	Download
Phototaxis.bsearch	extension	BehaviorSearch file for phototaxis (visual)	about 5 hours ago, by Eden Forbes	Download
Thigmotaxis.bsearch	extension	BehaviorSearch file for Thigmotaxis (tactile)	about 5 hours ago, by Eden Forbes	Download

This model does not have any ancestors.

This model does not have any descendants.

NetLogo