(yolo)=
# Object Detection with YOLO

[YOLO11](https://docs.ultralytics.com/) is a popular image detection framework
for Python. In this tutorial, you will see how we can implement an object
detection flow using xronos.

## Clone the YOLO example and install dependencies

Begin by cloning the `xronos` repository and navigating to the YOLO example.

```console
$ git clone https://github.com/xronos-inc/xronos
$ cd xronos/examples/YOLO
```

Set up a virtual environment and install xronos, see {ref}`xronos_installation` instructions.

Install the dependencies of the YOLOv11 example.

```console
$ apt-get install libgl1
$ pip install -r requirements.txt
```

## Overview

Familiarize yourself with the main program in `YOLO.py`. After parsing some
command line arguments, the `main()` function assembles the reactor program.

```{literalinclude} ../../examples/YOLO/YOLO.py
:pyobject: main
:dedent:
```

This creates the following reactor program.

![YOLO diagram](./_img/diagram_yolo.png)

In the following we will go through each of the three reactors.

## WebCam reactor
Open `WebCam.py`. The `WebCam` reactor declares two reactor elements, using an {class}`~xronos.ProgrammableTimerDeclaration` called
`_sample_webcam` and an {class}`~xronos.OutputPortDeclaration` called `frame`. Programmable timers are objects which the program
can use to schedule future events. Unlike periodic timers, where the timestamps of the events are periodic, programmable timers can be
scheduled at arbitrary intervals and are, as such, more expressive than periodic timers.
Reactors communicate by exchanging events via ports. In this case, `WebCam` can
send events containing `cv2` matrices.

```{literalinclude} ../../examples/YOLO/WebCam.py
:pyobject: WebCam
:dedent:
:end-before: def __init__
```

The `__init__` method defines the `sample_period` attribute and opens a `VideoCapture`
stream using `cv2`.

```{literalinclude} ../../examples/YOLO/WebCam.py
:pyobject: WebCam.__init__
:dedent:
```

The reactor contains three reactions. `on_startup` is triggered by
{attr}`~xronos.Reactor.startup`. The startup reaction uses
{func}`~xronos.ReactionInterface.add_effect` to declare an effect on the
programmable timer `_sample_webcam`. This allows the reaction handler to schedule
the first occurrence of `_sample_webcam` event. Instead of invoking
{func}`~xronos.ActionEffect.schedule` directly, the reaction handler uses the
convenience function `reschedule_sample_webcam` to schedule the first event. We
will explain this function in more detail later.

```{literalinclude} ../../examples/YOLO/WebCam.py
:pyobject: WebCam.on_startup
:dedent:
```

The `on_sample_webcam` reaction is triggered by the `_sample_webcam` event and may have
an effect on the `fame` output port. The main task of the handler is to read a
frame from the camera and to forward this frame to the next reactor using the
`frame` output port. A value is written to a port by calling {func}`~xronos.PortEffect.set`
on the declared port effect. If no frame could be read, the
handler stops the program using {func}`~xronos.Reactor.request_shutdown`.
Finally, the handler uses `reschedule_sample_webcam` to schedule the next
occurrence of `_sample_webcam`, so that the reaction will be triggered again in
the future.

```{literalinclude} ../../examples/YOLO/WebCam.py
:pyobject: WebCam.on_sample_webcam
:dedent:
```

The function `reschedule_sample_webcam`, that is used for scheduling new occurrences of the `_sample_webcam` event,
implements a simple adaptive load-balancing strategy. It uses {func}`xronos.Reactor.get_lag` to inspect how closely
events are processed with respect to real-time. If the lag is greater than `sample_period` we skip a number of
samples. Note that event itself is declared to carry no value. Therefore, it is scheduled using `value=None`.


```{literalinclude} ../../examples/YOLO/WebCam.py
:pyobject: WebCam.reschedule_sample_webcam
:dedent:
```

The `on_shutdown` reaction is triggered by the {attr}`~xronos.Reactor.shutdown`
event. The handler frees up resources and gracefully shuts the camera stream
down. Even in response to an unhandled exception or {kbd}`Ctrl`+{kbd}`C`, the
shutdown handler is invoked and closes the video stream.

```{literalinclude} ../../examples/YOLO/WebCam.py
:pyobject: WebCam.on_shutdown
:dedent:
```

## DNN reactor

The `DNN` reactor is found in `DNN.py`. It declares an input port called `frame` and output port called `result`.

```{literalinclude} ../../examples/YOLO/DNN.py
:pyobject: DNN
:dedent:
:end-before: def __init__
```

The `__init__` method checks whether there is a GPU with CUDA support and loads
the YOLO model parameters using the `ultralytics` module.

```{literalinclude} ../../examples/YOLO/DNN.py
:pyobject: DNN.__init__
:dedent:
```

The reactor defines a single reaction. It is triggered by the `frame` input
port and has the output port `result` as an effect. Upon receiving a frame it
does inference using the YOLO model and extracts bounding boxes, confidence
values and names. The result is written to the output port.

```{literalinclude} ../../examples/YOLO/DNN.py
:pyobject: DNN.on_frame
:dedent:
```

## Display reactor

The `Display` reactor is found in `Display.py`. It declares two input ports `frame` and `dnn_result`.

```{literalinclude} ../../examples/YOLO/Display.py
:pyobject: Display
:dedent:
:end-before: def __init__
```

The `__init__` method initializes a few attributes.
```{literalinclude} ../../examples/YOLO/Display.py
:pyobject: Display.__init__
:dedent:
```

The reactor defines two reactions. `on_frame` is triggered by both input ports.
This means that if a message is received on either input port, the reaction will
be executed. If both input ports receive a message with identical timestamps, the
xronos runtime ensures that the reaction is only executed once, with both input
ports present. In this case, the messages arrive *simultaneously*. The presence of
an event can be checked using the {func}`~xronos.Trigger.is_present` method. The ability
to align events on multiple input port simplifies the application logic needed.
The reaction handler can fully focus on sensor fusion.

```{literalinclude} ../../examples/YOLO/Display.py
:pyobject: Display.on_frame
:dedent:
```

Finally, the `on_shutdown` reaction is triggered by the
{attr}`xronos.Reactor.shutdown` event and closes the window.

```{literalinclude} ../../examples/YOLO/Display.py
:pyobject: Display.on_shutdown
:dedent:
```

## Bringing it all together
Run the example application with disabled debug logging from torch:
```console
$ YOLO_VERBOSE=False python YOLO.py
```
A window will appear displaying the video stream from your webcam and the object detection made by YOLO.

Optionally, you can append the `--telemetry` argument to visualize the program
execution in the {ref}`dashboard`.