Continuous-time recurrent neural network

2 collaborators

Eden Forbes (Author)

Gabriel Severino (Author)

WHAT IS IT?

This is an implementation of continuous-time recurrent neural networks (CTRNN) in NetLogo. CTRNNs have been used in a wide variety of applied research in biology and control theory. When coupled with certain simulated bodies and environments, search algorithms (such as evolutionary algorithms) can be used to find parameter sets for a CTRNN that accomplish some kind of task prescribed by the modeler.

HOW IT WORKS

CTRNNs are systems of continuous ordinary differential equations which here are approximated discretely (via 'ticks') using Euler integration. At each timestep, each node in a CTRNN is updated according to its current state, the state of connected nodes, and the strengths of connection between nodes, as well as some other inherent properties. The general equation for a CTRNN node is:

dy_i /dt = 1/T_i (-y_i + SUM(w_ij SIG(y_j + b_j) + I_i ))

where:

SIG denotes a standard sigmoidal firing curve: 1/(1 + e^-x)

b_i is a bias that determines the sensitive region of the sigmoidal firing curve for node i

T_i is a time constant which determines the maximum rate of change for node i

w_ij is a connection weight from node j to node i.

I_i summarizes external inputs to node i (say, via a sensor)

This implementation was built with the intention of being imported into larger models, such that each turtle in a simulation can have a CTRNN embedded within it which, in conjunction with inputs from the body and/or environment, shapes its behavior in some way. However, interacting with the base CTRNN model is a great way to understand the principles of CTRNNs: how they are constructed, how their parameters determine their activity, and how inputs change their activity.

HOW TO USE IT

The interface is fairly simple given this CTRNN implementation is intended as a submodel. However, we have provided several ways of playing with a CTRNN directly in the interface.

'Setup' initializes a 3-node CTRNN.

'CPG?' determines whether the initialized CTRNN has random parameters (if 'CPG?' is OFF) or if the CTRNN is initialized with a prescribed set of parameters that perpetually oscillate (a central pattern generato, if 'CPG?' is ON).

'runtime' simply determines the duration of simulation for the CTRNN.

'network-size' will set the network size. The first three nodes are initialized with the central pattern generator, while any additional nodes are random.

'external-input' determines the input to each node in the network. Here, we only provide one uniform input for all nodes, but unique inputs can easily be added by changing the setup function in the interface.

'Go' will simulate the behavior of the central pattern generator, plotting both the states of the CTRNN nodes and their outputs (normalized between 0 - 1).

THINGS TO NOTICE

Look at the relationships between correlations in behavior of node activity and the connection weights between each input!

THINGS TO TRY

Try changing certain parameter values slowly and see what effect they have on the behavior of the circuit.

Try changing the external input and see what effect it has on the behavior of the circuit.

EXTENDING THE MODEL

You can easily add or subtract nodes via the 'network-size' parameter. To see any additional nodes' activity (more than 3), just add more pens to the plots.

NETLOGO FEATURES

This model does not rely on turtles or patches at all (except for graphic representation of the networks). Instead, it simulates the system of differential equations that simulates a CTRNN while allowing the user to play with the parameters of a given network.

The NetLogo 'networks' extension is used to help facilitate this CTRNN implementation.

RELATED MODELS

Currently, this CTRNN implementation is directly imported into a partner model used to evolve foraging behaviors in NetLogo. However, CTRNNs could be attached to turtles in almost any model if the user writes procedures to couple the CTRNN's behavior to the behavior of turtles in the model in questino.

CREDITS AND REFERENCES

The CTRNN class in Netlogo was adapted from a C++ implementation originally designed by Dr. Randall Beer, which can be found at his homepage: https://rdbeer.pages.iu.edu/

Literature on the function and dynamics of CTRNNs:

Beer, R. D. (1995). On the dynamics of small continuous-time recurrent neural networks. Adaptive Behavior, 3(4), 469-509. https://doi.org/10.1177/105971239500300

Mathayomchan, B., & Beer, R. D. (2002). Center-crossing recurrent neural networks for the evolution of rhythmic behavior. Neural computation, 14(9), 2043-2051. https://doi.org/10.1162/089976602320263999

Beer, R. D. (2006). Parameter space structure of continuous-time recurrent neural networks. Neural computation, 18(12), 3009-3051. https://doi.org/10.1177/10597123950030040

Comments and Questions

Please start the discussion about this model! (You'll first need to log in.)

Click to Run Model

;; CTRNNs IN NETLOGO ;;
;; Eden Forbes 2025-26  ;;

extensions [nw]

globals [
  ctrnn-size
  ctrnn-states
  ctrnn-outputs
  ctrnn-biases
  ctrnn-gains
  ctrnn-taus
  ctrnn-Rtaus
  ctrnn-externalins
  ctrnn-weights
  ; RK4 temporaries
  ctrnn-temp-states
  ctrnn-temp-outputs
  ctrnn-k1
  ctrnn-k2
  ctrnn-k3
  ctrnn-k4
  neuron-turtles
]

breed [neurons neuron]

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Utility helpers for list mutation
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

;; set-item-in-list list idx value -> new-list

to-report set-item-in-list [lst idx val]
  report replace-item idx lst val
end 

;; n-values helper to make a list of n copies of value
;; (NetLogo has n-values built-in; using it directly below)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Sigmoid
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

to-report ctrnn-sigmoid [x]
  report 1 / (1 + exp (- x))
end 

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Initialize / Resize
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

to ctrnn-set-size [newsize]
  ;; set global size and initialize lists
  set ctrnn-size newsize
  set ctrnn-states (n-values ctrnn-size [0.0])
  set ctrnn-outputs (n-values ctrnn-size [0.0])
  set ctrnn-biases (n-values ctrnn-size [0.0])
  set ctrnn-gains (n-values ctrnn-size [1.0])
  set ctrnn-taus (n-values ctrnn-size [1.0])
  set ctrnn-Rtaus (n-values ctrnn-size [1.0])
  set ctrnn-externalins (n-values ctrnn-size [0.0])
  ;; weights: outer index = source neuron j, inner index = target i
  set ctrnn-weights (n-values ctrnn-size [ n-values ctrnn-size [0.0] ])
  ;; RK4 temps
  set ctrnn-temp-states (n-values ctrnn-size [0.0])
  set ctrnn-temp-outputs (n-values ctrnn-size [0.0])
  set ctrnn-k1 (n-values ctrnn-size [0.0])
  set ctrnn-k2 (n-values ctrnn-size [0.0])
  set ctrnn-k3 (n-values ctrnn-size [0.0])
  set ctrnn-k4 (n-values ctrnn-size [0.0])
end 

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Accessors / Mutators (optional helpers)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

to ctrnn-set-neuron-state [i val]  ;; i is 0-based index
  set ctrnn-states (set-item-in-list ctrnn-states i val)
end 

to-report ctrnn-neuron-state [src]
  report item src ctrnn-states
end 

to ctrnn-set-neuron-output [i val]
  set ctrnn-outputs (set-item-in-list ctrnn-outputs i val)
end 

to-report ctrnn-neuron-output [src]
  report item src ctrnn-outputs
end 

to ctrnn-set-neuron-bias [i val]
  set ctrnn-biases (set-item-in-list ctrnn-biases i val)
end 

to-report ctrnn-neuron-bias [src]
  report item src ctrnn-biases
end 

to ctrnn-set-neuron-tau [i val]
  set ctrnn-taus (set-item-in-list ctrnn-taus i val)
end 

to-report ctrnn-neuron-tau [src]
  report item src ctrnn-taus
end 

to ctrnn-set-connection-weight [src tgt val]  ;; src,tgt are 0-based indices
  let row (item src ctrnn-weights)
  set row (set-item-in-list row tgt val)
  set ctrnn-weights (set-item-in-list ctrnn-weights src row)
end 

to-report ctrnn-connection-weight [src tgt]
  report item tgt (item src ctrnn-weights)
end 

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Randomization
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

to ctrnn-randomize-states [lb ub]
  ;; uniform in [lb, ub)
  let rng-width (ub - lb)
  let new (map [ _ -> lb + random-float rng-width ] range ctrnn-size)
  set ctrnn-states new
end 

to ctrnn-randomize-outputs [lb ub]
  let rng-width (ub - lb)
  let new (map [ _ -> lb + random-float rng-width ] range ctrnn-size)
  set ctrnn-outputs new
end 

to randomize-ctrnn-parameters
  ; Randomize states, biases, gains, taus, weights
  ctrnn-randomize-states -1 1
  set ctrnn-biases (map [ _ -> random-float 2 - 1 ] range ctrnn-size)
  set ctrnn-gains (map [ _ -> 0.5 + random-float 2 ] range ctrnn-size)
  set ctrnn-taus (map [ _ -> 1 + random-float 2 ] range ctrnn-size)
  set ctrnn-Rtaus (map [ t -> 1 / t ] ctrnn-taus)
  
  ; Random weights between -2 and 2
  set ctrnn-weights (n-values ctrnn-size [
    n-values ctrnn-size [ -2 + random-float 4 ]
  ])
  
  ; zero external inputs for now
  set ctrnn-externalins (n-values ctrnn-size [0])
end 


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Integration - Euler
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

to ctrnn-euler-step [stepsize]
  ;; compute new states
  let new-states ctrnn-states
  ;; for each target neuron i
  foreach range ctrnn-size [
    i ->
    ;; compute input for neuron i
    let input item i ctrnn-externalins
    let j 0
    while [ j < ctrnn-size ] [
      let w (item j (item i ctrnn-weights)) ;;; HERE
      let outj item j ctrnn-outputs
      set input input + w * outj
      set j j + 1
    ]
    let state-i item i ctrnn-states
    let tau item i ctrnn-taus
    let newstate state-i + stepsize * tau * (input - state-i)
    set new-states (set-item-in-list new-states i newstate)
  ]
  set ctrnn-states new-states
  ;; update outputs
  let new-outs ctrnn-outputs
  foreach range ctrnn-size [
    i ->
    let s item i ctrnn-states
    let b item i ctrnn-biases
    let g item i ctrnn-gains
    let out (ctrnn-sigmoid (g * (s + b)))
    set new-outs (set-item-in-list new-outs i out)
  ]
  set ctrnn-outputs new-outs
end 

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Set center crossing
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

to ctrnn-set-center-crossing
  ;; For each target neuron i, sum input weights weights[j][i] across all j
  foreach range ctrnn-size [
    i ->
    let input-sum 0.0
    let j 0
    while [ j < ctrnn-size ] [
      set input-sum input-sum + (item i (item j ctrnn-weights))
      set j j + 1
    ]
    let theta-star (- input-sum / 2.0)
    set ctrnn-biases (set-item-in-list ctrnn-biases i theta-star)
  ]
end 

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Set CPG (two-neuron pattern)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

to ctrnn-set-cpg-2
  ;; N1 and N2 are 0-based indices
  ctrnn-set-neuron-bias 0 0.0
  ctrnn-set-neuron-bias 1 0.0
  ctrnn-set-connection-weight 0 0 -0.2
  ctrnn-set-connection-weight 0 1 -0.9
  ctrnn-set-connection-weight 1 0 -0.9
  ctrnn-set-connection-weight 1 1 -0.2
end 


; ==============================
; Visualization setup
; ==============================

to show-neurons
  ;; Layout radius
  let radius 200

  ;; Clear the world
  clear-turtles
  set neuron-turtles []

  ;; Create neurons evenly spaced around a circle
  foreach range ctrnn-size [ [i] ->
    ;; Ensure floating-point division by using 1.0
    let angle (2 * pi * 57.2958 * (i / (ctrnn-size)))
    show angle

    ;; Compute coordinates (NetLogo's y increases upward)
    let x radius * cos angle + 12
    let y radius * sin angle

    create-turtles 1 [
      setxy x y
      set shape "circle"
      set color green
      set size 5
      set label (word "N" i)
      set label-color white
      set neuron-turtles lput self neuron-turtles
    ]
  ]
end 

to show-weight-links
  ; delete existing links first
  ask links [ die ]
  
  ; make links colored by strength
  foreach range ctrnn-size [
    j ->
    foreach range ctrnn-size [
      i ->
      if j != i [
        let w (item i (item j ctrnn-weights))
        
         ask turtle j [
           create-link-to turtle i [
             set color 0.1 + abs w * 0.1
             set thickness 0.1 + abs w * 0.1
           ]
         ]
      ]
    ]
  ]
end 

to update-neuron-visuals
  ; update neuron color by current output
  ask turtles [
    let idx who
    let out item idx ctrnn-outputs
    set color scale-color green out 0 1
  ]
end 


; ==============================
; Setup and go
; ==============================

to setup
  clear-all
  ctrnn-set-size network-size
  
  ask patches [set pcolor white]
  
;  randomize-ctrnn-parameters
;  ctrnn-set-center-crossing
;  ctrnn-set-cpg-2
  
  ifelse CPG?[
    ;======================================;
    ; 3N CTRNN CPG From Chiel et al. 1999  ;
    ;======================================;
    ctrnn-set-neuron-tau 0 1 / 3.296257
    ctrnn-set-neuron-tau 1 1 / 3.852370
    ctrnn-set-neuron-tau 2 1 / 9.055600
    
    ctrnn-set-neuron-bias 0 0.550376
    ctrnn-set-neuron-bias 1 -1.549699
    ctrnn-set-neuron-bias 2 -13.250860
    
    ctrnn-set-connection-weight 0 0 5.236498
    ctrnn-set-connection-weight 0 1 7.226635
    ctrnn-set-connection-weight 0 2 -13.408114
    ctrnn-set-connection-weight 1 0 8.316531
    ctrnn-set-connection-weight 1 1 -1.288293
    ctrnn-set-connection-weight 1 2 -4.629253
    ctrnn-set-connection-weight 2 0 -2.985237
    ctrnn-set-connection-weight 2 1 14.669038
    ctrnn-set-connection-weight 2 2 9.630772
  ]
  [
    randomize-ctrnn-parameters
  ]
  
  ctrnn-randomize-states 0 1
  ctrnn-randomize-outputs 0 1
  set ctrnn-externalins (n-values ctrnn-size [external_input])
  
  show-neurons
  show-weight-links
  
  reset-ticks
end 

to go
  ; Example dynamics step (you can choose Euler or RK4)
  ctrnn-euler-step 0.05
  update-neuron-visuals
  if ticks > runtime [stop]
  tick
end

There is only one version of this model, created 3 days ago by Eden Forbes.

Attached files

File	Type	Description	Last updated
Continuous-time recurrent neural network.png	preview	Preview for 'Continuous-time recurrent neural network'	3 days ago, by Eden Forbes	Download

This model does not have any ancestors.

This model does not have any descendants.

NetLogo