RLEnvironment

Inherits: Spatial < Node < Object

Main class to manage communication and Godot app loop. User must override the class to set requered properties and initialization.

Description

This is top-level class that have RLAgent and RLEnvWorld as children nodes.

This class implements logic on:

Establishing connection between Godot app and client code (e.g. Python script).

2. Managing Godot app loop. The class pauses environment if there is no request from client to perform action to keep environemnt state while sending data.

3. Managing requests from client. The response is send only for observation request. Configuration request are not responded. Observation request is usually in conjunction with action request, forming environment step request.

Typical example of an project structure is:

Environment (inherited from RLEnvironment)
|___Agent (inherited from RLAgent)
|___World (inherited from RLEnvWorld)

Constants

CONFIG_KEY = "config"

RESET_KEY = "reset"

ACTION_KEY = "action"

OBSERVATION_KEY = "observation"

WORLD_KEY = "world"

AGENT_KEY = "agent"

ENVIRONMENT_KEY = "environment"

Property Descriptions

(RLAgent) agent: Agent node to be set by user.

(RLEnvWorld) world: World node to be set by user.

(Communication) communication: Communication server to handle request via TCP protocol.

(PhysicsFramesTimer) physics_frames_timer: Timer in terms of physics frames.

(int) repeat_action:: Amount of physics steps to repeat same action. Configurable from client.

Method Descriptions

(void) _enter_tree()

Note

The method is not expected to be overriden by user.

Called when the node enters the SceneTree.

The method disables physics on the environment launch.

(void) _ready()

Called when the node is “ready”, i.e. when both the node and its children have entered the scene tree.

User must override the method to set agent and world attributes and provide logic on server start. Example:

func _ready():
    world = $World
    agent = $Agent
    communication.start_server(9090, "127.0.0.1")

(void) _process(delta_: float)

Note

The method is not expected to be overriden by user.

Called during the processing step of the main loop.

Polls communication server to learn if there is any incomming requests.

Args:: delta_: Unused argument requred by the node parent class.

(void) _physics_process(delta_: float)

Note

The method is not expected to be overriden by user.

Called during the physics processing step of the main loop.

Increments physics_frames_timer counter.

Args:: delta_: Unused argument requred by the node parent class.

(void) _step(action)

Method to assign agent action and wait for repeat_action physics steps to be executed before pausing app again to emulate real-time interaction of client controlling agent and world.

The method can be overriden to provide additional logic as follows:

func _step(action):
    # some code here
    yield(._step(action), "completed")

or

func _step(action):
    # some code here
    agent.set_action(action)
        get_tree().set_pause(false)  # Enable physics
        yield(physics_frames_timer.start(), "timer_end")
    get_tree().set_pause(true)  # Disable physics

Args:: action: Any structure to be passed into RLAgent.set_action.

(void) _reset()

Method to be overriden by user to implement own logic on environment reset.

Generally, it is expected for logic of agent reset and world reset provided. E.g.:

func _reset():
    world.reset()
    agent.reset(world.sample_initial_position())

(void) _on_got_connection(request: Dictionary)

Note

The method is not expected to be overriden by user.

Method to handle incomming connection.

Args request: Dictionary with any combinations of RLEnvironment.RESET_KEY, RLEnvironment.CONFIG_KEY, RLEnvironment.AGENT_KEY, RLEnvironment.OBSERVATION_KEY keys.

(void) _send_response(observation_request: Dictionary)

Note

The method is not expected to be overriden by user.

Method to aggregate data from world and agent and send it.

Args:: observation_request: Dictionary with RLEnvironment.AGENT_KEY and RLEnvironment.WORLD_KEY keys.

(void) _after_send_response(): Optional method to implement logic on after sending response (e.g. cleaning recorded frames).

(void) _configure(configuration_request: Dictionary)

Note

Generally, the method is not expected to be overriden by user.

Method to manage agent, world and environment configuration.

Args:: configuration_request: Configuration with RLEnvironment.AGENT_KEY, RLEnvironment.WORLD_KEY, RLEnvironment.ENVIRONMENT_KEY keys.