diff --git a/README.md b/README.md index a1cf4a1..283e904 100644 --- a/README.md +++ b/README.md @@ -4,17 +4,57 @@ This repository contains the code for two research projects: 1. **MonStereo: When Monocular and Stereo Meet at the Tail of 3D Human Localization** [README](https://github.com/vita-epfl/monstereo/tree/master/docs/MonStereo.md) & [Article](https://arxiv.org/abs/2008.10913) - - ![monstereo](docs/out_005523.png) - - + + ![monstereo 1](docs/000840_multi.png) + 2. **Perceiving Humans: from Monocular 3D Localization to Social Distancing (MonoLoco++)** [README](https://github.com/vita-epfl/monstereo/tree/master/docs/MonoLoco_pp.md) & [Article](https://arxiv.org/abs/2009.00984) ![social distancing](docs/pull_sd.png) - ![monoloco](docs/truck.png) + ![monoloco_pp](docs/truck.png) -Both projects has been built upon [Openpifpaf](https://github.com/vita-epfl/openpifpaf) -for 2D pose estimation and [MonoLoco](https://github.com/vita-epfl/monoloco) for monocular 3D localization. -All projects share the AGPL Licence. \ No newline at end of file +Both projects has been built upon the CVPR'19 project [Openpifpaf](https://github.com/vita-epfl/openpifpaf) +for 2D pose estimation and the ICCV'19 project [MonoLoco](https://github.com/vita-epfl/monoloco) for monocular 3D localization. +All projects share the AGPL Licence. + + +# Setup +Installation steps are the same for both projects. + +### Install +The installation has been tested on OSX and Linux operating systems, with Python 3.6 or Python 3.7. +Packages have been installed with pip and virtual environments. +For quick installation, do not clone this repository, +and make sure there is no folder named monstereo in your current directory. +A GPU is not required, yet highly recommended for real-time performances. +MonStereo can be installed as a package, by: + +``` +pip3 install monstereo +``` + +For development of the monstereo source code itself, you need to clone this repository and then: +``` +pip3 install sdist +cd monstereo +python3 setup.py sdist bdist_wheel +pip3 install -e . +``` + +### Interfaces +All the commands are run through a main file called `main.py` using subparsers. +To check all the commands for the parser and the subparsers (including openpifpaf ones) run: + +* `python3 -m monstereo.run --help` +* `python3 -m monstereo.run predict --help` +* `python3 -m monstereo.run train --help` +* `python3 -m monstereo.run eval --help` +* `python3 -m monstereo.run prep --help` + +or check the file `monstereo/run.py` + +Further instructions for prediction, preprocessing, training and evaluation can be found here: + +* [MonStereo README](https://github.com/vita-epfl/monstereo/tree/master/docs/MonStereo.md) +* [MonoLoco++ README](https://github.com/vita-epfl/monstereo/tree/master/docs/MonoLoco_pp.md) diff --git a/docs/000840_multi.png b/docs/000840_multi.png new file mode 100644 index 0000000..d81ceaa Binary files /dev/null and b/docs/000840_multi.png differ diff --git a/docs/MonStereo.md b/docs/MonStereo.md index ae4b9f3..106ef20 100644 --- a/docs/MonStereo.md +++ b/docs/MonStereo.md @@ -24,53 +24,15 @@ month = {August}, year = {2020} } ``` + +# Prediction +The predict script receives an image (or an entire folder using glob expressions), +calls PifPaf for 2d human pose detection over the image +and runs MonStereo for 3d location of the detected poses. -# Features -The code has been built upon the ICCV'19 project [MonoLoco](https://github.com/vita-epfl/monoloco). -This repository supports +Output options include json files and/or visualization of the predictions on the image in *frontal mode*, +*birds-eye-view mode* or *multi mode* and can be specified with `--output_types` -* the original MonoLoco -* An improved Monocular version (MonoLoco++) for x,y,z coordinates, orientation, and dimensions -* MonStereo - -# Setup - -### Install -The installation has been tested on OSX and Linux operating systems, with Python 3.6 or Python 3.7. -Packages have been installed with pip and virtual environments. -For quick installation, do not clone this repository, -and make sure there is no folder named monstereo in your current directory. -A GPU is not required, yet highly recommended for real-time performances. -MonStereo can be installed as a package, by: - -``` -pip3 install monstereo -``` - -For development of the monstereo source code itself, you need to clone this repository and then: -``` -pip3 install sdist -cd monstereo -python3 setup.py sdist bdist_wheel -pip3 install -e . -``` - -### Data structure - - Data - ├── arrays - ├── models - ├── kitti - ├── logs - ├── output - - -Run the following to create the folders: -``` -mkdir data -cd data -mkdir arrays models kitti logs output -``` ### Pre-trained Models * Download Monstereo pre-trained model from @@ -85,27 +47,6 @@ Alternatively, you can download a Pifpaf pre-trained model from [openpifpaf](htt If you'd like to use an updated version, we suggest to re-train the MonStereo model as well. * The model for the experiments is provided in *data/models/ms-200710-1511.pkl* -# Interfaces -All the commands are run through a main file called `main.py` using subparsers. -To check all the commands for the parser and the subparsers (including openpifpaf ones) run: - -* `python3 -m monstereo.run --help` -* `python3 -m monstereo.run predict --help` -* `python3 -m monstereo.run train --help` -* `python3 -m monstereo.run eval --help` -* `python3 -m monstereo.run prep --help` - -or check the file `monstereo/run.py` - -# Prediction -The predict script receives an image (or an entire folder using glob expressions), -calls PifPaf for 2d human pose detection over the image -and runs MonStereo for 3d location of the detected poses. - - -Output options include json files and/or visualization of the predictions on the image in *frontal mode*, -*birds-eye-view mode* or *multi mode* and can be specified with `--output_types` - ### Ground truth matching * In case you provide a ground-truth json file to compare the predictions of MonSter, @@ -139,6 +80,22 @@ but require first to run a pose detector over all the training images and collect the annotations. The code supports this option (by running the predict script and using `--mode pifpaf`). +### Data structure + + Data + ├── arrays + ├── models + ├── kitti + ├── logs + ├── output + +Run the following to create the folders: +``` +mkdir data +cd data +mkdir arrays models kitti logs output +``` + ### Datasets Download KITTI ground truth files and camera calibration matrices for training