english.knit

class: middle

<div>
<style type="text/css">.xaringan-extra-logo {
width: 300px;
height: 128px;
z-index: 0;
background-image: url(figs/logo.png);
background-size: contain;
background-repeat: no-repeat;
position: absolute;
bottom:1em;left:88%;
}
</style>
<script>(function () {
  let tries = 0
  function addLogo () {
    if (typeof slideshow === 'undefined') {
      tries += 1
      if (tries < 10) {
        setTimeout(addLogo, 100)
      }
    } else {
      document.querySelectorAll('.remark-slide-content:not(.title-slide):not(.inverse):not(.hide_logo)')
        .forEach(function (slide) {
          const logo = document.createElement('div')
          logo.classList = 'xaringan-extra-logo'
          logo.href = null
          slide.appendChild(logo)
        })
    }
  }
  document.addEventListener('DOMContentLoaded', addLogo)
})()</script>
</div>

<br>
<br>
<center>
# Creating open, reproducible workflows for ecological niche modeling

### Andrea Sánchez-Tapia, Sara Ribeiro Mortara, Felipe Sodré Barros

### FOSS4G Buenos Aires 2021
</center>
<br>

__[Diapositivas en Castellano](https://andreasancheztapia.github.io/FOSS4G_2021/espa%C3%B1ol#1)__

---
background-image: url("figs/logo_jbrj.png")
background-position: 98% 2%
background-size: 100px
.left-column[
<img src="figs/andrea.jpg" title="Pictures of the team: Andrea Sánchez-Tapia, Sara Mortara, Felipe Sodré Barros, Guilherme Gall, Diogo Rocha, and Marinez Ferreira de Siqueira (PI)" alt="Pictures of the team: Andrea Sánchez-Tapia, Sara Mortara, Felipe Sodré Barros, Guilherme Gall, Diogo Rocha, and Marinez Ferreira de Siqueira (PI)" width="90" style="display: block; margin: auto auto auto 0;" /><img src="figs/SRM.jpg" title="Pictures of the team: Andrea Sánchez-Tapia, Sara Mortara, Felipe Sodré Barros, Guilherme Gall, Diogo Rocha, and Marinez Ferreira de Siqueira (PI)" alt="Pictures of the team: Andrea Sánchez-Tapia, Sara Mortara, Felipe Sodré Barros, Guilherme Gall, Diogo Rocha, and Marinez Ferreira de Siqueira (PI)" width="90" style="display: block; margin: auto auto auto 0;" /><img src="figs/Felipe_Barros.jpg" title="Pictures of the team: Andrea Sánchez-Tapia, Sara Mortara, Felipe Sodré Barros, Guilherme Gall, Diogo Rocha, and Marinez Ferreira de Siqueira (PI)" alt="Pictures of the team: Andrea Sánchez-Tapia, Sara Mortara, Felipe Sodré Barros, Guilherme Gall, Diogo Rocha, and Marinez Ferreira de Siqueira (PI)" width="90" style="display: block; margin: auto auto auto 0;" /><img src="figs/gui.jpeg" title="Pictures of the team: Andrea Sánchez-Tapia, Sara Mortara, Felipe Sodré Barros, Guilherme Gall, Diogo Rocha, and Marinez Ferreira de Siqueira (PI)" alt="Pictures of the team: Andrea Sánchez-Tapia, Sara Mortara, Felipe Sodré Barros, Guilherme Gall, Diogo Rocha, and Marinez Ferreira de Siqueira (PI)" width="90" style="display: block; margin: auto auto auto 0;" /><img src="figs/diogo.jpeg" title="Pictures of the team: Andrea Sánchez-Tapia, Sara Mortara, Felipe Sodré Barros, Guilherme Gall, Diogo Rocha, and Marinez Ferreira de Siqueira (PI)" alt="Pictures of the team: Andrea Sánchez-Tapia, Sara Mortara, Felipe Sodré Barros, Guilherme Gall, Diogo Rocha, and Marinez Ferreira de Siqueira (PI)" width="90" style="display: block; margin: auto auto auto 0;" /><img src="figs/mari.jpeg" title="Pictures of the team: Andrea Sánchez-Tapia, Sara Mortara, Felipe Sodré Barros, Guilherme Gall, Diogo Rocha, and Marinez Ferreira de Siqueira (PI)" alt="Pictures of the team: Andrea Sánchez-Tapia, Sara Mortara, Felipe Sodré Barros, Guilherme Gall, Diogo Rocha, and Marinez Ferreira de Siqueira (PI)" width="90" style="display: block; margin: auto auto auto 0;" />
]

.right-column[
#### Scientific Computation Laboratory - Rio de Janeiro Botanical Garden
#### ¡liibre! Independent Biodiversity Informatics Laboratory

+ Scientific workflows based in R for data downloading and cleaning, taxonomic checking
+ Biodiversity informatics, ENM/SDM, open science, reproducibility
+ Support for data-intensive research projects (e.g., CNCFlora - IUCN authority)
]

---
background-image: url("https://the-turing-way.netlify.app/_static/logo.jpg")
background-position: 4% 4%
background-size: 200px

.right[
### https://the-turing-way.netlify.app/welcome
]

<br>
<br>
<br>
<br>
+ [Open research](https://the-turing-way.netlify.app/reproducible-research/open.html): reproducible, transparent, reusable, collaborative, accountable, accessible to society
+ [Reproducibility](https://the-turing-way.netlify.app/reproducible-research/reproducible-research.html):
data and code being available to fully rerun the analysis

---
## ecological niche modeling or species distribution modeling (ENM, SDM)

+ __Stastistical learning__ applied to the prediction of areas potentially suitable for the presence of a species

+ Species occurrence data (collection data bases, GBIF)

+ Environmental predictor variables  (Worldclim, remote sensing)

+ Diferent algorithms (GLM, SVM, random forests, classification trees, GBM)

+ Model performance (data partition into training and test data + performance metrics)

+ Early implementations: hidden code ("black boxes"), point-and-click applications

---
background-image: url("https://www.r-project.org/Rlogo.png")
background-position: 4% 4%
background-size: 200px

- Statistical programming language
--

- Libre software, open code, _lingua franca_ in Ecology
--

- Several options for ENM/SDM in R
--

- GIS with __raster__, __sp__, __maps__, __rgdal__, __sf__
--

- Established packages such as __dismo__ (Hijmans et al 2017), __BIOMOD2__ (Thuiller et al 2007)
--

- Recent packages: **ENMeval** (Muscarella et al 2014), **sdm** (Naimi & Araujo 2016), **spThin** (Aiello-Lammens et al 2015), **zoon** (Golding et al. 2018), **wallace** (Kass et al 2018), **kuenm** (Cobos et al 2019), **occCite** (Lowens 2020)
--

#### Difficulties with scalability, code repetition, decision documentation... .purple[non-reproducible]

---
background-image: url("figs/modleR.png")
background-position: 4% 4%
background-size: 150px

- https://model-r.github.io/modleR/index.html
--

A .purple[workflow] developed to __integrate and automatize__ some of the common steps in ecological niche modeling
--

- a four-step workflow:
--

+ `setup_sdmdata()`: data setup, cleaning, partition, pseudoabsence selection
--

+ `do_any()` and `do_many()`: model fitting, projecting and evaluating
--

+ `final_model()`: joining partitions
--

+ `ensemble_model()`: algorithm consensus

---
background-image: url("figs/modleR.png")
background-position: 4% 4%
background-size: 150px

.right[
## a four-step workflow

<img src="figs/WORKFLOW.png" title="Workflow scheme. The first step cleans and partitions data into training and test sets, samples pseudoabsences and controls for the correlation of explanatory variables. The second step fits an algorithm per partition, for each train and test set. The third step joins partitions, creating a model per algorithm. The fourth step analyzes algorithm consensus -'ensemble modeling', to obtain a unique model per species" alt="Workflow scheme. The first step cleans and partitions data into training and test sets, samples pseudoabsences and controls for the correlation of explanatory variables. The second step fits an algorithm per partition, for each train and test set. The third step joins partitions, creating a model per algorithm. The fourth step analyzes algorithm consensus -'ensemble modeling', to obtain a unique model per species" width="1000" />
]

---
## 1. `setup_sdmdata()`: data preparation

+ Data cleaning: exact duplicates, NAs and one occurrence per pixel (this does not exclude the need for previous careful data cleaning!)

+ Experimental design: bootstrap, cross-validation

+ Pseudo-absence sampling

+ Control of variable correlation up to a user-defined value (e.g., 0.8)

---
## 2. `do_(m)any()`: model fitting and projection

+ `do_any()` for one algorithm and partition (ex. `algo = "maxent"`)

+ `do_many()` calls `do_any()` to fit multiple algorithms (ex: `bioclim = TRUE`, `maxent = TRUE`)

+ Parametrization
--

- Algorithms: dismo, maxnet, GLM, SVM, bosques aleatorios, boosted regression trees (BRT)

+ Projection to different datasets (in time or space)

+ Returns table with performance statistics -> TSS, AUC, pROC, FNR, Jaccard...

---
## 2. `do_(m)any()`: model fitting and projection

_Abarema langsdorffii_, three partitions, randomForests

---
## 3. `final_model()`: a model per algorithm per species

+ The basics: a central tendency measure and uncertainty between partitions

+ Which models to join? (the raw continuous model, the binary)

+ Some additional operations: consensus between binary models

+ Uncertainty: range (max - min) between partitions

---
## 3. `final_model()`: a model per algorithm per species

1. Raw mean
2. Binary consensus
3. Uncertainty (range)

---
## 4. `ensemble_model()` - algorithmic consensus

+ Mean between final models

+ Consensus

+ Best-performing algorithm

+ PCA between algorithms

+ Range uncertainty metrics

+ Ensemble models do not necessarily perform better than individual algorithms (Zhu & Peterson 2017)

+ Evaluate different ensemble model performance (WIP)

---
class: center, middle

# modleR was written to implement good practices in scientific computation

---
## folder structure and portability

.pull-left[

+ A single working directory per project

+ Different steps: different subfolders

+ A consistent subfolder structure

+ Relative rather than absolute paths and no `setwd()`

]

.pull-right[
<img src="figs/fig03_folder.png" width="772" />
]

---
## modularity

+ Each step saves its output

+ The next step reads the previous output

+ Using HD space rather than RAM:

+ The user may enter and exit the workflow at any step
--

+ Parallelization and use in high performance/high throughput computational frameworks (HPC/HTC)

---
## thorough metadata recording

+ Metadata: parametrization options

+ Session information

+ Environment: packages used and their version

---
## interoperability

We did not create new classes or methods: communication with other packages in the R environment

---
## reproducibility

---
## final remarks

+ Reproducibility should drive any ENM workflow

+ __Metadata__ are really useful - and necessary

+ __Scalability__: Any ENM workflow should be easily adapted to HPC

+ __Flexibility__ to start and leave at any step is essential to guarantee the communication with other packages and the requirements of different projects

+ Part of a larger group of R packages in the area: efforts for __integration__

---
class: middle, center

# ¡Gracias!