add KI
This commit is contained in:
121
README.md
121
README.md
@@ -1,89 +1,46 @@
|
||||
# Tello Drone Sim
|
||||
# Tello AI Pilot (Extreme Performance Edition)
|
||||
|
||||
This is a simple simulation of a Tello drone using Ursina. The drone can be controlled via tcp calls.
|
||||
Dieses Projekt ist eine hochoptimierte KI-Steuerung für die DJI Tello Drohne (und den integrierten Simulator). Es nutzt modernste Computer-Vision-Modelle via ONNX Runtime, um Personen zu verfolgen, Hindernissen auszuweichen und Ziele intelligent wiederzuerkennen.
|
||||
|
||||
In the repo there is the simulation server along with a client class that can be used to interact with the sim server
|
||||
## 🚀 Hauptmerkmale
|
||||
- **Hybrid AI Engine**: Nutzt **YuNet** (Face Detection), **MiDaS** (Depth Estimation) und **MobileNetV3** (ReID) – alle via **ONNX Runtime** für maximale FPS.
|
||||
- **Multithreading**: Kamera-Stream und KI-Verarbeitung laufen in getrennten Threads. Das Videobild bleibt flüssig, egal wie schwer die KI arbeitet.
|
||||
- **Visual Fingerprinting (ReID)**: Die Drohne erstellt einen digitalen Fingerabdruck einer fixierten Person und findet sie automatisch wieder, wenn sie kurz aus dem Bild verschwindet.
|
||||
- **Intelligente Verfolgung**: Aktive Suche in Verschwindungsrichtung und aggressiver Verfolgungsmodus.
|
||||
- **Echtzeit-HUD**: Professionelles Display mit Telemetrie, AI-Tiefenkarte und Radar-Zonen zur Hindernisvermeidung.
|
||||
- **Rate Limiting**: Kontrollierter Datenfluss (10Hz) zum Tello-SDK für maximale Verbindungsstabilität.
|
||||
|
||||

|
||||
## 🕹 Steuerung
|
||||
Klicke mit der **Maus** direkt in das Videobild, um eine Person zu fixieren (**Lock-ON**).
|
||||
|
||||
## Setup
|
||||
### Tastatur-Belegung:
|
||||
| Taste | Aktion |
|
||||
| :--- | :--- |
|
||||
| **T** | **Takeoff** (Starten) - Einmal drücken und kurz warten |
|
||||
| **L** | **Land** (Landen) |
|
||||
| **M** | Wechsel zwischen **Manuell** und **KI-Modus** |
|
||||
| **K** | Lock-ON Trigger aktivieren / Fixierung löschen |
|
||||
| **Space**| **Not-Aus** (Stoppt alle Motoren sofort) |
|
||||
| **W/S** | Vorwärts / Rückwärts (Manuell) |
|
||||
| **A/D** | Links / Rechts (Manuell) |
|
||||
| **R/F** | Steigen / Sinken (Manuell) |
|
||||
| **E/Z** | Drehen Links / Rechts (Manuell) |
|
||||
| **1** | Automatisches Drehen (Scan-Modus) an/aus |
|
||||
| **Enter**| Programm beenden |
|
||||
|
||||
### Option 1: Dev Container (Recommended)
|
||||
## 🏗 Architektur
|
||||
- `drone_pilot/main.py`: Das Herzstück. Verwaltet Threads, UI-Events und die Hauptschleife.
|
||||
- `drone_pilot/vision.py`: Die KI-Engine. Lädt ONNX-Modelle und verarbeitet Bilddaten.
|
||||
- `drone_pilot/flight.py`: Der Flug-Controller. Berechnet RC-Vektoren basierend auf KI-Ergebnissen.
|
||||
- `drone_pilot/ui.py`: HUD-Renderer für das OpenCV-Fenster.
|
||||
- `drone_pilot/config.py`: Zentrale Konfiguration für Geschwindigkeiten und Schwellenwerte.
|
||||
|
||||
The easiest way to get started is using the provided dev container which includes all dependencies and GUI support:
|
||||
## 🛠 Installation & Start
|
||||
1. Installiere die Abhängigkeiten: `pip install -r requirements.txt`
|
||||
2. Stelle sicher, dass die Modelle im Ordner `models/` liegen.
|
||||
3. Starte das Programm:
|
||||
- Simulator: `python run.py` (Default: `use_real_tello=False`)
|
||||
- Reale Drohne: Ändere in `run.py` zu `use_real_tello=True`
|
||||
|
||||
1. **Setup the dev container for your platform:**
|
||||
|
||||
```bash
|
||||
.devcontainer/setup.sh
|
||||
```
|
||||
|
||||
This will auto-detect your platform (macOS, Linux, Windows, WSL) and generate the appropriate `devcontainer.json`.
|
||||
|
||||
2. **Open in VS Code:**
|
||||
- Install the "Dev Containers" extension
|
||||
- Open Command Palette (Cmd/Ctrl + Shift + P)
|
||||
- Run "Dev Containers: Reopen in Container"
|
||||
|
||||
3. **Platform-specific requirements:**
|
||||
- **macOS**: Install XQuartz (`brew install --cask xquartz`) and run `xhost +localhost`
|
||||
- **Linux**: X11 forwarding should work out of the box
|
||||
- **Windows**: Access GUI via VNC at `http://localhost:5901` (password: `vncpass`)
|
||||
|
||||
### Option 2: Manual Setup
|
||||
|
||||
If you prefer to set up the environment manually:
|
||||
|
||||
1. Create the virtual environment by running:
|
||||
|
||||
```bash
|
||||
python3.12 -m venv venv
|
||||
```
|
||||
|
||||
2. Activate the virtual environment by running:
|
||||
|
||||
```bash
|
||||
source venv/bin/activate
|
||||
```
|
||||
|
||||
3. Install the required packages by running:
|
||||
|
||||
```bash
|
||||
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org -r requirements.txt
|
||||
```
|
||||
|
||||
4. Export the python path by running:
|
||||
|
||||
```bash
|
||||
export PYTHONPATH=$PWD
|
||||
```
|
||||
|
||||
## Running the simulation
|
||||
|
||||
To run the simulation, run the following command:
|
||||
|
||||
```bash
|
||||
python tello_sim/run_sim.py
|
||||
```
|
||||
|
||||
You can try running some of the [examples](./examples) to see how the simulation works. The examples are located in the `examples` folder.
|
||||
|
||||
Or use the [client](./tello_sim_client.py) class to interact with the simulation server. The client class is located in the `tello_sim` folder.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- For a specific python version on macOS, consider using [pyenv](https://github.com/pyenv/pyenv?tab=readme-ov-file#b-set-up-your-shell-environment-for-pyenv) to manage multiple python versions.
|
||||
- Another alternative for macOS users is to use [Homebrew](https://brew.sh/) to install the desired python version:
|
||||
|
||||
```bash
|
||||
brew install python@3.12
|
||||
```
|
||||
|
||||
- Conda users can create an environment with the desired python version:
|
||||
|
||||
```bash
|
||||
conda create -n tello-sim python=3.12
|
||||
conda activate tello-sim
|
||||
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org -r requirements.txt
|
||||
export PYTHONPATH=$PWD
|
||||
```
|
||||
---
|
||||
*Hinweis: Beim Fliegen mit der realen Drohne immer auf einen Akkustand > 15% achten, da der Startbefehl sonst vom SDK abgelehnt wird.*
|
||||
|
||||
Reference in New Issue
Block a user