Description

The ncrmp.benthic.analysis package is designed to use benthic coral reef data collected from the Atlantic regions of NOAA’s National Coral Reef Monitoring Program (NCRMP) to complete the following:

  1. Compute summary statistics (e.g. coral density, percent cover, species richness, old and recent mortality, disease prevalence, Diadema density) from the site to the regional level as well as presence/absence of ESA species and bleaching abundance at the strata and site levels, respectively.

  2. House all current NCRMP benthic data ranging from analysis ready data to regional domain estimates in a standardized format. This includes all metrics listed in item #1 calculated at the site, strata and regional level (as applicable) as well as the most recent sampling grids and NTOT files. There are over 250 dataframes stored in this package.

Note:

For questions, please contact the repository owner, Shay Viehman,

NCRMP Regions and sampling years

  • Florida:
    • Southeast Florida (SEFCRI; 2014, 2016, 2018, 2020, 2022)
    • Florida Keys (FLK; 2014, 2016, 2018, 2020*, 2022)
    • Dry Tortugas (Tortugas; 2014, 2016, 2018, 2020, 2022)
  • Caribbean:
    • St. Thomas & St. John (STTSTJ; 2013, 2015, 2017, 2019, 2021, 2023)
    • St. Croix (STX; 2015, 2017, 2019, 2021, 2023)
    • Puerto Rico (PRICO; 2014, 2016, 2019, 2021, 2023)
  • Gulf of America:
    • Flower Garden Banks National Marine Sanctuary (FGB; 2013, 2015, 2018, 2022, 2024)

Upcoming data additions: Florida Regions 2024 (in 2025)

Installation

  1. Download the zipped R package using the green “Clone or download” button. Unzip the file.

  2. In R studio, open the ncrmp.benthics.analysis R project in the folder of the same name and use the following code to install the pacakge. You must be in the in the ncrmp.benthics.analysis R project. If you aleady have devtools installed you can skip the first line.

install.packages('devtools')

devtools::install()
  1. Exit the ncrmp.benthics.analysis R project and check your R library to confirm the package installed. At this point you can either delete the files from the zipped folder or retain them if you are interested in the analysis code.

  2. From your current project, use the following code to load and use the package.

library(ncrmp.benthics.analysis) 
  1. To update the package (when new data is added), repeat steps 1-4. If you are having trouble, try deleting the old version first before intalling the new one.

Note: Requires R version 3.4.1 or higher.

Currently, you cannot load the package directly from Git Hub using devtools::install_github(‘shgroves/NCRMP_benthics’). Future updates will inlcude this feature.


To use this package as a data repository

To use this package as a data repository requires very little experience with R! You will need to have R version 3.4.1 or higher installed on your computer as well as R Studio desktop. Please see https://cran.r-project.org/doc/manuals/r-release/R-admin.html and https://www.rstudio.com/products/rstudio/download/ for more information.

Accessing Data

Since all datasets are stored within the package, you can either use the dataset directly in your code or assign it to an object in your Global Environment.

Add dataframe to the Global Environment

require(ncrmp.benthics.analysis)
#> Loading required package: ncrmp.benthics.analysis

#The 'data' function will load a dataframe to the Global Environment keeping the existing name,

data("NCRMP_DRM_FLK_2014_22_density_site")

#or you can give it a new name

dat <- NCRMP_DRM_FLK_2014_22_density_site

If you need to use a dataframe outside of R, you can export it as a CSV file using the write.csv() function.

Export a dataframe

# require(ncrmp.benthics.analysis)
# write.csv(NCRMP_DRM_FLK_2014_18_density_site, "Your file path\\NCRMP_DRM_FLK_14_16_density_site.csv", row.names = F)

Use a dataframe in a ggplot

require(ncrmp.benthics.analysis)
require(ggplot2)

# Load weighted strata means
Hardcoral <- NCRMP_STTSTJ_2013_23_cover_region %>%
  dplyr::ungroup() %>%
  # filter to just hardcoral
  dplyr::filter(cover_group == "HARD CORALS") %>%
  # Change Year to factor so R sees it as a grouping variable and not a numberd
  dplyr::mutate(YEAR = as.factor(YEAR))

# Create your plot
ggplot(Hardcoral, aes(x = YEAR, y = avCvr, group = YEAR)) +
  geom_bar(aes(fill = YEAR), stat="identity", colour="black",
           position=position_dodge(),
           size=.3) +
# Add error bars
  geom_errorbar(aes(ymax = avCvr+SE, ymin = avCvr-SE), width = .5, position=position_dodge(1)) +
# Set the y axis scale
  scale_y_continuous(limits = c(0,25), expand = c(0, 0)) +
# Add axis labels
  xlab("Year") +
  ylab("Cover (%)") +
  labs(title = "Hard coral cover") +
# Remove legend
  theme(legend.position="none")

Datasets

Datasets contained within the package at the strata and regional level are created using the default function parameters:

  • For Florida regions:
    • project = “NCRMP” (NCRMP only) or “NCRMP_DRM” (NCRMP and DRM combined)
    • species filter = FALSE (filter not applied)
  • For all Carib regions:
    • project = “NCRMP”
    • species filter = FALSE
  • For FGB:
    • project = “NCRMP”
    • species filter = FALSE

Analysis Ready (AR) data

Analysis ready NCRMP datasets for all regions and years are contained within this package in the most recent NCRMP Atlantic archive format.

Dataset Naming Convention:

Datasets are structured as follows:

REGION_YEAR_datatype

Where:

  • REGION = regional code (ex: “SEFCRI” or “FLK”)

  • YEAR = Year of data collection

  • datatype = benthic_cover, coral_demographics, or inverts_ESAcorals

Two-stage (2-transect) data are labeled explicitly except for Tortugas 2018, where NCRMP and DRM data were combined. Tortugas 2018 coral demographics surveys were a joint mission between NCRMP and DRM where two transects were conducted at each site. The NCRMP and DRM data are combined and indistinguishable. All other datasets are single transect.

  • Examples:
    • FLK_2016_coral_demographics,
    • SEFCRI_2014_2stage_benthic_cover
    • FGBNMS_2013_inverts_ESAcorals

Additionally, two datasets aggregate all regions and years:

  • NCRMP_benthic_cover_All_regions_years

  • NCRMP_demo_All_regions_years

Florida DRM and SCREAM Data

In addition, the following AR Florida DRM and Sanctuary Coral Reef Ecosystem Assessment Monitoring (SCREAM) datasets in NCRMP format are stored in the package. Combined DRM and SCREAM data are from 1999 to 2013 (not all regions are sampled each year) and individual DRM regions are 2014 to the most recent year of sampling.

  • preNCRMP_SCREAM_DRM_coral_demographics
  • DRM_SEFCRI_2014_2022_2stage_coral_demographics- includes data from 2014-2022
  • DRM_FLK_2014_2022_2stage_coral_demographics- includes data from 2014-2022
  • DRM_Tort_2014_2022_2stage_coral_demographics- includes data from 2014-2022

Summary dataframes

Summary dataframes are provided for site, strata and weighted regional means. These dataframes also follow a standardized naming convention.

Dataset Naming Convention:

PROJECT_REGION_YEAR_datatype_level

Where:

  • PROJECT = NCRMP or NCRMP_DRM

  • REGION = Regional code ex: “PRICO” or “FLK”

  • YEARs = time range, ex: 2014_22 or 2013_22

  • datatype = data type, ex: dis_prev or density

  • level = site, strata, region, domain

Examples:

  • NCRMP_DRM_FLK_2014_22_dis_prev_site,

  • NCRMP_FGBNMS_2013_22_density_strata,

  • NCRMP_FLK_2014_22_rec_mort_region,

  • NCRMP_PRICO_2014_21_richness_strata,

  • NCRMP_STX_2015_21_cover_region

In addition, there are three dataframes that contain all regions and years

  • NCRMP_AllRegions_Years_ESA_PresAbs_Site
  • NCRMP_AllRegions_Years_ESA_PresAbs_Strat
  • NCRMP_bleaching_abundance_Allyears_Allregions

Shapefiles and NTOT files

The most recent sampling grid shapefiles used for site allocation and all NTOT (grid summary) files are also contained in the package. Grids can be used in R or exported as shapefiles.

Sampling grids:

  • SEFCRI_2018_sample_frame
  • FLK_2020_coral_demographics
  • Tort_2020_sample_frame
  • STTSTJ_2021_sample_frame_WGS1984
  • STX_2021_sample_frame_WGS1984
  • PRICO_2021_sample_frame_WGS1984
  • FGBNMS_2018_sample_frame_WGS1984

The NTOT files are required to properly apply the weighting scheme to summary data and often differ between years.

Available NTOT files:

  • USVI_2021_NTOT (STTSTJ & STX)
  • FLK: FLK_2014_NTOT, FLK_2016_NTOT, FLK_2018_NTOT, FLK_2020_NTOT, FLK_2022_NTOT
  • SEFCRI: SEFL_2014_NTOT, SEFL_2016_NTOT, SEFL_2018_NTOT, SEFL_2020_NTOT, SEFL_2020_NTOT
  • Tortugas: Tort_2014_NTOT, Tort_2016_NTOT, Tort_2018_NTOT, Tort_2020_NTOT, Tort_2022_NTOT
  • PRICO_2021_NTOT
  • FGBNMS_2022_NTOT

The Florida Keys National Marine Sanctuary (FKNMS) and the Flower Garden Banks National Marine Sanctuary (FGBNMS) boundary polygons are also contained in the package. More boundary polygons for other regions will be added in future updates.

  • Boundaries
    • FKNMS_boundary
    • FGBNMS_boundary

Additional files

  • ncrmp_frrp_sppcodes: species code, species names and benthic categories for NCRMP and other Florida monitoring programs.

To use the functions in this package

If you would like to run the AR data through the functions, apply preset species filters to certain functions or just know more about how NCRMP metrics are calculated, please feel free to use these functions or check out the code. Comments or suggestions on how to improve the package are welcome.

Functions for specific regions

These functions generate outputs for a single region and either a single year or multiple sampling years. They apply a stratified random sampling weighting scheme to compute metrics at the regional (sampling domain) level.

Benthic cover

NCRMP_calculate_cover

Purpose: Calculates percent cover from species to regional level for individual years

  • parameters:
    • region: “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • project: “NCRMP”, or “MIR” Note - MIR option is in development.

  • outputs (dataframes):
    • cover_group_key: a region specific key for species codes and major benthic categories

    • percent_cover_species: site level percent cover by species

    • percent_cover_site: site level percent cover by major benthic category

    • cover_strata: strata level mean percent cover of major benthic categories (unweighted)

    • Domain estimates: weighted regional (domain level) percent cover of major benthic categories

    • Domain_check: QAQC check that sums the regional percent cover. Should equal ~100

    • cover_check_site: QAQC check that sums the percent cover for each site. Should equal 100

  • Gets Called by:
    • NCRMP_make_weighted_LPI_data
    • load_NCRMP_benthic_cover_data

Example:

# 
# require(ncrmp.benthics.analysis)
# 
# # Run function and assign names to dataframes 
# tmp <- NCRMP_calculate_cover(region = "SEFCRI", project = "NCRMP")
# 
# # The function will store all of the function outputs as a list in tmp
# # Convert list objects to dataframes and rename 
# NCRMP_SEFCRI_2014_22_percent_cover_species <- tmp$percent_cover_species
# NCRMP_SEFCRI_2014_22_percent_cover_site <- tmp$percent_cover_site
# NCRMP_SEFCRI_2014_22_cover_group_key <- tmp$cover_group_key
# NCRMP_SEFCRI_2014_22_cover_strata <- tmp$cover_strata
# NCRMP_SEFCRI_2014_22_cover_region <- tmp$Domain_est
# NCRMP_SEFCRI_2014_22_Domain_check <- tmp$Domain_check 
# cover_check_site <- tmp$cover_check_site
# 
# # OR add them all at once into your global environment
#
# list2env(tmp, envir = .GlobalEnv)

NCRMP_colony_percent_cover

Purpose: This function calculates coral cover strata means and regional weighted means by species. It also produces a plot of percent cover by species for the most recent year of data available for the region

  • parameters:
    • region (string): “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • project (string): “NCRMP”, or “MIR” Note - MIR option is in development

    • file_path (string): file path where the output figure should be stored

    • ptitle (string): title of the plot

  • outputs (dataframes):
    • region_means
    • strata_means
    • plot (jpeg file)

Gets Called by:

NCRMP_make_weighted_species_coral_cover_data

load_NCRMP_benthic_cover_data

Purpose: load and combine benthic cover data for a given project (NCRMP or MIR) and region

  • parameters:
    • region: “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • project: “NCRMP”, or “MIR” Note - MIR option is in development.

  • outputs (dataframes):
    • dat (dataframe): containing the combined benthic cover data

NCRMP_make_weighted_LPI_data

Purpose: Applies weighting scheme to benthic cover data

Strata Definitions by Region:

  • Caribbean (USVI, PRICO): Strata are defined by HABITAT CODE + DEPTH STRAT.

  • Gulf of America (FGB): Only one strata (FGBNMS) is used, meaning no weighting is applied.

  • Florida:

    • SEFCRI: Strata are defined by STRAT (no PROT).

    • FLK: PROT is always set to 0 because benthic sites are not allocated at the PROT level.

    • Tortugas: PROT is maintained as is.

This function is called within NCRMP_calculate_cover, meaning the region and year parameters you provide there will be automatically passed on.

Example of weighting application for Florida:


#Calculate average cover (avcvr), stratum variance (svar), n and std at the strata and regional level using STRAT + PROT as the analysis strata. Inputdata = 1 stage site level cover for major benthic categories and ntot = the most recent NTOT file for the region, regardless of sampling year. 

# cover_est <- inputdata %>%
#    # group by analysis level strata
#   dplyr::mutate(ANALYSIS_STRATUM = paste(STRAT, "/ PROT =", PROT, sep = " ")) %>%
#   dplyr::group_by(ANALYSIS_STRATUM, STRAT, PROT, cover_group) %>% 
#   dplyr::summarise(
#     # calculate mean cover
#     avcvr = mean(Percent_Cvr),
#     # calculate stratum variance
#     svar = var(Percent_Cvr),
#     # calculate N
#     n = sum(n),
#     # calculate mean stratum depth
#     MIN_DEPTH = mean(MIN_DEPTH),
#     MAX_DEPTH = mean(MAX_DEPTH),
#     DEPTH_M = (MIN_DEPTH+MAX_DEPTH)/2) %>%
#   
#   # convert 0 for stratum variance so that the sqrt is a small # but not a 0
#   dplyr::mutate(svar = dplyr::case_when(svar == 0 ~ 0.00000001,
#                                         TRUE ~ svar)) %>%
#   dplyr::mutate(std = sqrt(svar))
# 
# cover_est <- cover_est %>%
#   # Merge ntot with cover_est
#   dplyr::full_join(., ntot) %>%
#   # stratum estimates
#   dplyr::mutate(whavcvr = wh * avcvr,
#                 whsvar = wh^2 * svar,
#                 whstd = wh * std,
#                 n = tidyr::replace_na(n, 0))
# 
# # Reformat data
# 
# # cover, unweighted by strata
# cover_strata <- cover_est %>%
#   dplyr::select(REGION, YEAR, ANALYSIS_STRATUM, STRAT, RUG_CD, PROT, DEPTH_M, NTOT, ngrtot, wh, cover_group, n, avcvr, svar, std) %>%
#   dplyr::mutate(METHOD = "UNWEIGHTED") %>%
#   dplyr::mutate(n = tidyr::replace_na(n, 0))
# 
# # Calculate Domain Estimates
#   Domain_est <- cover_est %>%
#     dplyr::group_by(REGION, YEAR, cover_group) %>%
#     dplyr::summarise(avCvr = sum(whavcvr),
#                      var = sum(whsvar, na.rm = T),
#                      std = sqrt(var),
#                      ngrtot = sum(NTOT) )

Example of weighting application for Caribbean and FGB:


# Calculate average cover (avcvr), stratum variance (svar), n and std at the strata and regional level using HABITAT + DEPTH_STRAT as the analysis strata. Inputdata = 1 stage site level cover for major benthic categories and ntot = the most recent NTOT file for the region, regardless of sampling year. 
# 
# cover_est <- inputdata %>%
#   # group by analysis level strata
#   dplyr::group_by(YEAR, ANALYSIS_STRATUM, STRAT, cover_group) %>% 
#   dplyr::summarise(
#     # compute average cover
#     avcvr = mean(Percent_Cvr),
#     # compute stratum variance
#     svar = var(Percent_Cvr),
#     # calculate N
#     n = sum(n),
#     # calculate mean stratum depth
#     MIN_DEPTH = mean(MIN_DEPTH),
#     MAX_DEPTH = mean(MAX_DEPTH),
#     DEPTH_M = (MIN_DEPTH+MAX_DEPTH)/2) %>%
#   # convert 0 for stratum variance so that the sqrt is a small # but not a 0
#   dplyr::mutate(svar = dplyr::case_when(svar == 0 ~ 0.00000001,
#                                         TRUE ~ svar)) %>%
#   dplyr::mutate(std = sqrt(svar))
# 
# cover_est <- cover_est %>%
#   # Merge ntot with cover_est
#   dplyr::full_join(., ntot) %>%
#   # stratum estimates
#   dplyr::mutate(whavcvr = wh * avcvr,
#                 whsvar = wh^2 * svar,
#                 whstd = wh * std,
#                 n = tidyr::replace_na(n, 0),
#                 # Add the following to match FL format
#                 PROT = NA,
#                 RUG_CD = NA)
# 
# # Reformat data
# 
# # cover, unweighted by strata
# cover_strata <- cover_est %>%
#   dplyr::select(REGION, YEAR, ANALYSIS_STRATUM, STRAT, RUG_CD, PROT, DEPTH_M, NTOT, ngrtot, wh, cover_group, n, avcvr, svar, std) %>%
#   dplyr::mutate(METHOD = "UNWEIGHTED") %>%
#   dplyr::mutate(n = tidyr::replace_na(n, 0))
# 
# 
# ## Calculate Domain Estimates
# Domain_est <- cover_est %>%
#   dplyr::group_by(REGION, YEAR, cover_group) %>%
#   dplyr::summarise(avCvr = sum(whavcvr),
#                    var = sum(whsvar, na.rm = T),
#                    std = sqrt(var),
#                    ngrtot = sum(NTOT) )
#

NCRMP_DRM_calculate_colony_density

Purpose: Calculates colony density from site to regional level for all sampling years

  • parameters:

    • project: “NCRMP” (All regions), “NCRMP_DRM” (Florida regions only)

    • region: “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • species_filter: A string indicating whether a region-specific preset species filter should be applied to density calculations (species codes listed) or all species should be included (FALSE; the default).

  • outputs (dataframes):

    • density_species: site level coral density by species

    • density_site: site level coral density

    • density_strata: strata level coral density (unweighted)

    • Domain estimates: weighted regional (domain level) coral density

Associated functions: NCRMP_make_weighted_demo_data, load_NCRMP_DRM_demo_data

Example:


# require(ncrmp.benthics.analysis)
# 
# # Run function 
# tmp <- NCRMP_DRM_calculate_colony_density(project = "NCRMP_DRM", 
#                                            region = "FLK",
#                                            species_filter = "FALSE")
# 
# # The function will store all of the function outputs as a list in tmp
# # Convert list objects to dataframes and rename 
# 
# NCRMP_DRM_FLK_2014_22_density_species <- tmp$density_species
# NCRMP_DRM_FLK_2014_22_density_site <- tmp$density_site
# NCRMP_DRM_FLK_2014_22_density_strata <- tmp$density_strata
# NCRMP_DRM_FLK_2014_22_density_region <- tmp$Domain_est
#
# # OR add them all at once into your global environment
#
# list2env(tmp, envir = .GlobalEnv)

NCRMP_DRM_colony_density_CV_and_occurrence

Calculates colony density and coefficient of variation (CV) by species at the regional level for all sampling years. Also produces a barplot of occurrence and CV by species for the most recent years data

  • parameters:

    • project: “NCRMP” (All regions), “NCRMP_DRM” (Florida regions only)

    • region: “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • species_filter: A string indicating whether a region-specific preset species filter should be applied to density calculations (species codes listed) or all species should be included (FALSE; the default).

    • year: a numeric indicating the year of interest for plotting, usually the most recent year

    • ptitle: a string indicating the figure title

    • file_path: a string indicating the file path to store the produced figure

  • outputs (dataframes):

    • region_means: weighted regional (domain level) coral density by species

    • region_means_cv20: weighted regional (domain level) coral density by species for species with a CV of 20 or less

Associated functions: NCRMP_make_weighted_demo_data, load_NCRMP_DRM_demo_data, tech memo Rmds

Example:


# require(ncrmp.benthics.analysis)
# 
# # Run function 
# tmp <- NCRMP_DRM_colony_density_CV_and_occurrence(region = "Tortugas",
#                                               ptitle = "Dry Tortugas",
#                                               year = 2022,
#                                               file_path = "K:/_BioGeoProjects/NCRMP/Data Analysis/Analysis ready data/Figures" ,  #must be connected to VPN if filepath is to a network drive
#                                               species_filter = "NULL",
#                                               project = "NCRMP")
# 
# 
# # The function will store all of the function outputs as a list in tmp
# # Convert list objects to dataframes and rename 
# 
# NCRMP_Tort_Occ_Den_CV <- tmp$region_means
# NCRMP_Tort_Occ_Den_CV_20 <- tmp$region_means_cv20

NCRMP_make_weighted_density_CV_data

Purpose: calculates regional weighted mean coral density and coefficient of variation (CV) for different coral species

  • parameters:

    • project (string): “NCRMP” (All regions), “NCRMP_DRM” (Florida regions only)

    • region (string): “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • sppdens (dataframe): dataframe containing coral density data

  • outputs (dataframes):

    • region_means: regional weighted means of coral density & CV

NCRMP_make_weighted_demo_data

Applies weighting scheme to coral demographic data (coral density, species richness, mortality, disease/bleaching prevalence).

Strata Definitions by Region:

  • Caribbean (USVI, PRICO): Strata are defined by HABITAT CODE + DEPTH STRAT.

  • Gulf of America (FGB): Only one strata (FGBNMS) is used, meaning no weighting is applied.

  • Florida:

    • SEFCRI: Strata are defined by STRAT (no PROT).

    • FLK: PROT is always set to 0 because benthic sites are not allocated at the PROT level.

    • Tortugas: PROT is maintained as is.

This function is called within NCRMP_DRM_calculate_colony_density so the initial parameters you provide for region and year will be passed on.

Example of weighting application for Florida:


# Calculate average density (avden), stratum variance (svar), n and std at the strata and regional level using STRAT + PROT as the analysis strata. Inputdata = 1 stage site level coral density and ntot = the most recent NTOT file for the region, regardless of sampling year. 

# density_est <- inputdata %>%
#   # group by analysis level strata
#   dplyr::mutate(ANALYSIS_STRATUM = paste(STRAT, "/ PROT =", PROT, sep = " ")) %>%
#   dplyr::group_by(YEAR, ANALYSIS_STRATUM, STRAT, PROT) %>% 
#   dplyr::summarise(
#     # compute average density
#     avden = mean(DENSITY),
#     # compute stratum variance
#     svar = var(DENSITY),
#     # calculate N
#     n = length(DENSITY)) %>%
#   # convert 0 for stratum variance so that the sqrt is a small # but not a 0
#   dplyr::mutate(svar = dplyr::case_when(svar == 0 ~ 0.00000001,
#                                         TRUE ~ svar)) %>%
#   dplyr::mutate(std = sqrt(svar))
# 
# density_est <- density_est %>%
#   # Merge ntot with coral_est_spp
#   dplyr::full_join(., ntot) %>%
#   # stratum estimates
#   dplyr::mutate(whavden = wh * avden,
#                 whsvar = wh^2 * svar,
#                 whstd = wh * std,
#                 n = tidyr::replace_na(n, 0))
# 
# # Reformat output
# 
# density_strata <-  density_est %>%
#   dplyr::select(REGION, YEAR, ANALYSIS_STRATUM, STRAT, RUG_CD, PROT, NTOT, ngrtot, wh, n, avden, svar, std)
# 
# # Calculate Domain Estimates
# Domain_est <- density_est %>%
#   dplyr::group_by(REGION, YEAR) %>%
#   dplyr::summarise(avDen = sum(whavden),
#                    var = sum(whsvar, na.rm = T),
#                    std = sqrt(var),
#                    ngrtot = sum(NTOT) )

Example of weighting application for Caribbean:


# Calculate average density (avden), stratum variance (svar), n and std at the strata and regional level using HABITAT + DEPTH_STRAT as the analysis strata. Inputdata = 1 stage site level coral density and ntot = the most recent NTOT file for the region, regardless of sampling year. 
# 
# # Calculate avdns, svar, n and std
# density_est <- inputdata %>%
#   # group by analysis level strata
#   dplyr::mutate(ANALYSIS_STRATUM = STRAT) %>%
#   dplyr::group_by(YEAR, ANALYSIS_STRATUM, STRAT) %>% 
#   dplyr::summarise(# compute average density
#     avden = mean(DENSITY),
#     # compute stratum variance
#     svar = var(DENSITY),
#     # calculate N
#     n = length(DENSITY)) %>%
#   # convert 0 for stratum variance so that the sqrt is a small # but not a 0
#   dplyr::mutate(svar = dplyr::case_when(svar == 0 ~ 0.00000001,
#                                         TRUE ~ svar)) %>%
#   dplyr::mutate(std = sqrt(svar))
# 
# density_est <- density_est %>%
#   # Merge ntot with coral_est_spp
#   dplyr::full_join(., ntot) %>%
#   # stratum estimates
#   dplyr::mutate(whavden = wh * avden,
#                 whsvar = wh^2 * svar,
#                 whstd = wh * std,
#                 n = tidyr::replace_na(n, 0),
#                 # Add the following to match FL format 
#                 PROT = NA,           
#                 RUG_CD = NA)
# 
# # Reformat output
# 
# density_strata <-  density_est %>%
#   dplyr::select(REGION, YEAR, ANALYSIS_STRATUM, STRAT, RUG_CD, PROT, NTOT, ngrtot, wh, n, avden, svar, std)
# 
# # Calculate Domain Estimates
# Domain_est <- density_est %>%
#   dplyr::group_by(REGION, YEAR) %>%
#   dplyr::summarise(avDen = sum(whavden),
#                    var = sum(whsvar, na.rm = T),
#                    std = sqrt(var),
#                    ngrtot = sum(NTOT) )

NCRMP_make_weighted_species_coral_cover_data

Purpose: Calculates weighted mean density and coefficient of variation (CV) for coral cover data by species

  • Inputs

    • project: “NCRMP_DRM” (Florida regions only), “NCRMP” (All regions)

    • region: “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • sppcvr (dataframe): dataframe containing site- and species-level percent cover data

  • Outputs:

    • region_means (dataframe): regional weighted mean cover by coral species

    • strata_means (dataframe): dataframe with strata-specific mean cover for coral species,

NCRMP_DRM_calculate_species_richness_diversity

Calculates species richness from site to regional level for all sampling years. Calculates Simpson, Inverse Simpson and Shannon–Wiener diversity indices from site to regional level for all sampling years using the R package vegan.

  • parameters:

    • project: “NCRMP” (All regions), “NCRMP_DRM” (Florida regions only)

    • region: “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

  • outputs (dataframes):

    • species_list: a list of a species present for all sampling years

    • richness_site: site level species richness. Includes juveniles. Note there is no accouting for transect length/effort

    • richness_strata: strata level mean species richness (unweighted). Includes juveniles. Note there is no accouting for transect length/effort

    • Domain estimates: weighted regional (domain level) species richness. Includes juveniles. Note there is no accouting for transect length/effort

    • species_diversity_site: site level species diversity using the Simpson, Inverse Simpson and Shannon-Weiner diversity indices. Based only on adults since the abundance of juveniles is not recorded. Note there is no accouting for transect length/effort

    • species_diversity_strata: strata level species diversity using the Simpson, Inverse Simpson and Shannon-Weiner diversity indices (unweighted). Based only on adults since the abundance of juveniles is not recorded. Note there is no accouting for transect length/effort

    • species_diversity_region: weighted regional (domain level) species diversity using the Simpson, Inverse Simpson and Shannon-Weiner diversity indices. Based only on adults since the abundance of juveniles is not recorded. Note there is no accouting for transect length/effort

Associated functions: NCRMP_make_weighted_demo_data

Example:


# require(ncrmp.benthics.analysis)
# 
# # Run function 
# tmp <- NCRMP_DRM_calculate_species_richness_diversity(project = "NCRMP",
#                                                        region= "Tortugas")
# 
# # The function will store all of the function outputs as a list in tmp
# # Convert list objects to dataframes and rename 
# 
# NCRMP_Tort_2014_22_spp_list <- tmp$species_list 
# NCRMP_Tort_2014_22_richness_site <- tmp$richness_site
# NCRMP_Tort_2014_22_richness_strata <- tmp$richness_strata
# NCRMP_Tort_2014_22_richness_region <- tmp$Domain_est
# 
# NCRMP_Tort_2014_22_diversity_site <- tmp$species_diversity_site
# NCRMP_Tort_2014_22_diversity_strata <- tmp$diversity_strata
# NCRMP_Tort_2014_22_diversity_region <- tmp$Domain_est_div
#
# # OR add them all at once into your global environment
#
# list2env(tmp, envir = .GlobalEnv)

Regional species richness and diversity are calculated using the same weighting scheme as coral density. See NCRMP_make_weighted_demo_data and NCRMP_make_weighted_demo_data_RC for example code. Only the default analysis strat settings are available for species diversity. See https://cran.r-project.org/web/packages/vegan/vegan.pdf for more information on the diversity indices.

NCRMP_DRM_calculate_mortality

Calculates old and recent mortality (percent of colony surface) from species to regional level for all sampling years

  • parameters:

    • project: “NCRMP” (All regions), “NCRMP_DRM” (Florida regions only)

    • region: “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • species_filter: A string indicating whether a region-specific preset species filter should be applied to density calculations (species codes in a list) or all species should be included (FALSE; the default). Generally not used as species-specific estimates are also provided by default.

  • outputs (dataframes):

    • old_mortality_site: site level old mortality (across all corals)

    • recent_mortality_site: site level recent mortality (across all corals)

    • old_mortality_strata: strata level mean old mortality (unweighted, across all corals)

    • rec_mortality_strata: strata level mean recent mortality (unweighted, across all corals)

    • old_mortality_species_strata: strata level mean old mortality (unweighted, by species)

    • rec_mortality_species_strata: strata level mean recent mortality (unweighted, by species)

    • Domain_est_old_mort: weighted regional (domain level) old mortality (across all species)

    • Domain_est_rec_mort: weighted regional (domain level) recent mortality (across all species)

    • Domain_est_old_mort_species: weighted regional (domain level) old mortality, by species. Not species-specific mortality weighting is based on strata where species is PRESENT

    • Domain_est_rec_mort_species: weighted regional (domain level) recent mortality, by species. Not species-specific mortality weighting is based on strata where species is PRESENT

    • ntot_check_rec_mort_species: QAQC check to ensure species-specific NTOT’s add up to 1

    • ntot_check_old_mort_species: QAQC check to ensure species-specific NTOT’s add up to 1

Associated functions: NCRMP_make_weighted_demo_data

Example:


# require(ncrmp.benthics.analysis)
# 
# # Run function
# tmp <- NCRMP_DRM_calculate_mortality(project = "NCRMP",
#                                       region = "FGB",
#                                       species_filter = "FALSE")
# 
# # The function will store all of the function outputs as a list in tmp
# # Convert list objects to dataframes and rename
# 
# NCRMP_FGBNMS_2013_22_old_mort_site <- tmp$old_mortality_site
# NCRMP_FGBNMS_2013_22_rec_mort_site <- tmp$recent_mortality_site
# 
# NCRMP_FGBNMS_2013_22_old_mort_strata <- tmp$old_mortality_strata
# NCRMP_FGBNMS_2013_22_rec_mort_strata <- tmp$rec_mortality_strata
# 
# NCRMP_FGBNMS_2013_22_old_mort_region <- tmp$Domain_est_old_mort
# NCRMP_FGBNMS_2013_22_rec_mort_region <- tmp$Domain_est_rec_mort
# 
# NCRMP_FGBNMS_2013_22_old_mort_species_region <- tmp$Domain_est_old_mort_species
# NCRMP_FGBNMS_2013_22_rec_mort_species_region <- tmp$Domain_est_rec_mort_species
#
#
# # check ntot weighting for species-specific mortality - should only be values of 1 (sum of weights across strata for each species in each year)
# unique(tmp$ntot_check_rec_mort_species$wh_sum)
# unique(tmp$ntot_check_old_mort_species$wh_sum)

Regional old and recent mortality are calculated using the same weighting scheme as coral density. See NCRMP_make_weighted_demo_data for example code. Species-specific mortality estimates are done based on strata a species is PRESENT. See load_NTOT_species for code.

Invertebrate density

NCRMP_calculate_invert_density

Calculates Diadema antillarum density from site to regional level for all sampling years

  • parameters:

    • region: “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • project: “NCRMP” (default) or “MIR” (in development)

  • outputs (dataframes):

    • diadema_density_site: site level Diadema density

    • invert_strata: strata level mean Diadema density (unweighted)

    • Domain_est: weighted regional (domain level) Diadema density

Associated functions: NCRMP_make_weighted_invert_density

Example:

# 
# require(ncrmp.benthics.analysis)
# 
# 
# # Run function 
# tmp <- NCRMP_calculate_invert_density(region = "PRICO", project = "NCRMP")
# 
# # The function will store all of the function outputs as a list in tmp
# # Convert list objects to dataframes and rename 
# 
# NCRMP_PRICO_2014_21_invert_den_site <- tmp$diadema_density_site
# NCRMP_PRICO_2014_21_invert_den_strata <- tmp$invert_strata # Right now just Diadema
# NCRMP_PRICO_2014_21_invert_den_region <- tmp$Domain_est

Regional Diadema density is calculated using the same weighting scheme as coral density. See NCRMP_make_weighted_invert_density for example code. This function is currently only calculating Diadema density. Future updates will include conch and lobster; but until these updates are complete, please see the code in this function for guidance on how to calculate density for the other invertebrates.

NCRMP_calculate_invert_density

Calculates The purpose of this function is to calculate weighted invertebrate density data for a benthic assessment project. This function is used to calculate weighted strata means and regional estimates of invertebrate density

  • parameters:

    • inputdata: dataframe with site level invert density data

    • region: “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • project: “NCRMP” (default) or “MIR” (in development)

  • outputs (dataframes):

    • invert_strata: Dataframe containing invertebrate density information for each stratum, including weighted and unweighted values

    • Domain_est: Dataframe containing the weighted regional estimates for invertebrate density, summarizing by region and year.

Associated functions: NCRMP_calculate_invert_density

NCRMP_DRM_calculate_disease_prevalence_colonies

Purposes: Calculates disease and bleaching prevalence at the site (% colonies per site), site and species (% colonies per site per species), strata (mean of % colonies per site), and regional (weighted mean of % colonies per site) level for all sampling years

  • parameters:

    • project: “NCRMP_DRM” (Florida regions only), “NCRMP” (All regions)

    • region: “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • species_filter: A string indicating whether a region-specific preset species filter should be applied to density calculations (species codes in a list) or all species should be included (FALSE; the default). Note the species filter is not used as much as species-specific domain estimates are also provided.

  • outputs (dataframes):

    • dis_ble_prev_species: site and species level disease and bleaching prevalence

    • dis_ble_prev_site: site level disease and bleaching prevalence

    • dis_prev_strata: strata level disease prevalence (unweighted)

    • ble_prev_strata: strata level bleaching prevalence (unweighted)

    • Domain_est: weighted regional (domain level) disease and bleaching prevalence

Associated functions: NCRMP_make_weighted_demo_data

Example:


# require(ncrmp.benthics.analysis)
# 
# # Run function 
# 
# tmp <- NCRMP_DRM_calculate_disease_prevalence_colonies(project = "NCRMP_DRM",
#                                                        region = "SEFCRI",
#                                                        species_filter = "FALSE")
# 
# # The function will store all of the function outputs as a list in tmp
# # Convert list objects to dataframes and rename 
# 
# NCRMP_DRM_SEFCRI_2014_22_dis_ble_prev_species <- tmp$dis_ble_prev_species
# NCRMP_DRM_SEFCRI_2014_22_dis_ble_prev_site <- tmp$dis_ble_prev_site
# NCRMP_DRM_SEFCRI_2014_22_dis_prev_strata <- tmp$dis_prev_strata
# NCRMP_DRM_SEFCRI_2014_22_ble_prev_strata <- tmp$ble_prev_strata
# NCRMP_DRM_SEFCRI_2014_22_dis_ble_prev_region <- tmp$Domain_est

Regional disease prevalence are calculated using the same weighting scheme as coral density. See NCRMP_make_weighted_demo_data for example code.

Note: Disease was not sampled in the Flower Garden Banks National Marine Sanctuary (FGBNMS) in 2013

NCRMP_DRM_calculate_dis_ble_prevalence_species_domain

Purpose: Calculates disease and bleaching prevalence by species at the regional (weighted mean of % colonies per site) level for all sampling years. Note weighting is specific to the strata each species is present in, because disease prevalence is NA (not 0) in a strata where a species is absent

  • parameters:

    • project: “NCRMP_DRM” (Florida regions only), “NCRMP” (All regions)

    • region: “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • species_filter: A string indicating whether a region-specific preset species filter should be applied to density calculations (species codes in a list) or all species should be included (FALSE; the default). Note the species filter is not used as much as species-specific domain estimates are also provided.

  • outputs (dataframes):

    • DomainEst_dis: weighted regional (domain level) disease prevalence by species

    • DomainEst_bl: weighted regional (domain level) bleaching prevalence by species

    • wh_total: QAQC check to ensure species-specific NTOT’s add up to 1

Associated functions: NCRMP_make_weighted_demo_data

Example:


# require(ncrmp.benthics.analysis)
# 
# # Run function 
# 
# tmp <- NCRMP_DRM_calculate_dis_ble_prevalence_species_domain(project="NCRMP",
#                                                              region = "STX")
# # ntot check - wh_total should always be 1 (sum of weights across strata for each species in each year)
# unique(tmp$ntot_check$wh_total)
# 
# # The function will store all of the function outputs as a list in tmp
# # Convert list objects to dataframes and rename 
# 
# NCRMP_STX_2015_21_dis_prev_species_region <- tmp$DomainEst_dis
# NCRMP_STX_2015_21_ble_prev_species_region <- tmp$DomainEst_bl

NCRMP_DRM_calculate_disease_prevalence_sites

Purpose: Calculates number of sites with disease and bleaching at the strata and regional level for all sampling years

  • parameters:

    • project: “NCRMP_DRM” (Florida regions only), “NCRMP” (All regions)

    • region: “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • species_filter: A string indicating whether a region-specific preset species filter should be applied to density calculations (species codes in a list) or all species should be included (FALSE; the default). Note the species filter is not used as much as species-specific domain estimates are also provided.

  • outputs (dataframes):

    • dis_ble_prev_site: site level disease and bleaching presence

    • dis_ble_prev_strata: number of sites in each strata with disease and bleaching

    • disease_prev_region: number of sites across the region with disease and bleaching

Associated functions: NCRMP_make_weighted_demo_data

Example:

# 
# require(ncrmp.benthics.analysis)
# 
# # Run function 
# 
# tmp <- NCRMP_DRM_calculate_disease_prevalence_sites(project = "NCRMP",
#                                                     region = "STTSTJ",
#                                                     species_filter = "FALSE")
# 
# # The function will store all of the function outputs as a list in tmp
# # Convert list objects to dataframes and rename 
# 
# NCRMP_STTSTJ_2013_21_dis_ble_PA_site <- tmp$dis_ble_prev_site
# NCRMP_STTSTJ_2013_21_dis_n_sites_per_strata <- tmp$dis_ble_prev_strata
# NCRMP_STTSTJ_2013_21_dis_ble_n_sites_per_region <- tmp$disease_prev_region

load_NTOT

Purpose: process and load NTOT (Total Coral Population) data for specific regions and projects, calculating the corresponding weighting scheme for coral demographic data

  • parameters:

    • project: “NCRMP_DRM” (Florida regions only), “NCRMP” (All regions)

    • region: “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • inputdata: dataframe containing coral demographic data,

  • outputs (dataframes):

    • ntot (dataframe): calculated weighted scheme (ntot) for the inputted region

load_NTOT_species

Purpose: loads NTOT data, calculates weighted demographic values for species based on their presence in different strata

  • parameters:

    • project: “NCRMP_DRM” (Florida regions only), “NCRMP” (All regions)

    • region: “SEFCRI”, “FLK”, “Tortugas”, “STTSTJ”, “STX”, “PRICO”, “FGB”

    • inputdata: dataframe containing stratum weights for each species, year, and region

  • outputs (dataframes):

    • ntot (dataframe): dataframe containing the weighted demographic data for species across regions and years

Functions for all regions and years

These functions create outputs that contain all regions and all years. They are not weighted.

ESA listed corals - presence absence

NCRMP_calculate_ESA_corals_PresAbs

Purpose: Calculates strata and site level presence / absence for all regions and sampling years. Currently NCRMP data only.

  • parameters:

    • None
  • outputs (dataframes):

    • NCRMP_AllRegions_Years_ESA_PresAbs_Strat: strata level ESA presence/absence

    • NCRMP_AllRegions_Years_ESA_PresAbs_Site: site level ESA presence/absence

    • region_year: a list to check that every region and year is included

Associated functions: None

Example:

# require(ncrmp.benthics.analysis)
# 
# # Run function 
# tmp <- NCRMP_calculate_ESA_corals_PresAbs()
# 
# # The function will store all of the function outputs as a list in tmp
# # Convert list objects to dataframes and rename 
# 
# NCRMP_AllRegions_Years_ESA_PresAbs_Strat <- tmp$NCRMP_AllRegions_Years_ESA_PresAbs_Strat
# NCRMP_AllRegions_Years_ESA_PresAbs_Site <- tmp$NCRMP_AllRegions_Years_ESA_PresAbs_Site
# 
# region_year <- tmp$region_year

NCRMP_make_ESA_corals_FOI

Purpose: Calculates regional ESA frequency of incidence, across all ESA species and for each, for all regions and sampling years based on the number of sites ESA are present. Currently NCRMP data only.

  • parameters:

    • None
  • outputs (dataframes):

    • FOI_region: regional frequency of incidence across all ESA corals for all regions and years

    • FOI_species: regional frequency of incidence of ESA corals by species for all regions and years

Associated functions: None

Example:

# require(ncrmp.benthics.analysis)
# 
# # Run function 
# tmp <- NCRMP_make_ESA_corals_FOI()
# 
# # The function will store all of the function outputs as a list in tmp
# # Convert list objects to dataframes and rename 
# NCRMP_ESA_FOI_region <- tmp$FOI_region
# NCRMP_ESA_FOI_species <- tmp$FOI_species

Bleaching abundance

NCRMP_make_bleaching_abundance

Purpose: Calculates species level bleaching abundance by each transect for all regions and sampling years. Currently NCRMP data only

  • parameters:

    • None
  • outputs (dataframes):

    • NCRMP_bleaching_abundance_Allyears_Allregions: species level bleaching abundance for each transect

Example:

# require(ncrmp.benthics.analysis)
# 
# # Run function 
# 
# tmp <- NCRMP_make_bleaching_abundance()
# 
# # The function will store all of the function outputs as a list in tmp
# # Convert list objects to dataframes and rename 
# 
# NCRMP_bleaching_abundance_Allyears_Allregions <- tmp$NCRMP_bleaching_abundance_Allyears_Allregions

New Functions

Update_FLK_Protection_Status

Purpose: updates the protection status (PROT) in FLK dataset by merging it with a reference grid (grid_df)

  • parameters:

    • data (florida dataset that needs updating)
    • grid_df (florida reference grid)
  • outputs (dataframes):

    • updated dataframe with updated florida protection status

Associated Functions: load_NCRMP_DRM_demo_data

GAM_function

Purpose: Designed to fit a Generalized Additive Model (GAM) to dataset, assess its statistical significance, and perform model diagnostics

  • parameters:

    • dataset with the associated variable

    • response: the dependent variable (must be a column in data)

    • group_var (optional): a grouping variable for subsetting the data

    • group_names (optional): a subset of group levels to include (ex: Macroalgae and Hard Coral)

    • kvalue: the smoothing parameter (k) that controls the flexibility (default = 3)

  • outputs (dataframes):

    • Model summary: list with key statistics and diagnostic results

    • diagnostic plots: histogram of residuals, Q-Q plot to check for normality, residuals vs fitted values plot for homoscedasticity check

    Associated Functions: Tech Memo Rmd


Frequently asked questions:

1. What species are used when you filter the species in NCRMP_DRM_calculate_colony_density, NCRMP_DRM_calculate_disease_prevalence or NCRMP_DRM_calculate_mortality?

NCRMP species filter developed in status report meetings

Report_card_filter <- c(“ACR CERV”, “ACR PALM”, “COL NATA”, “DIP LABY”, “MON CAVE”, “ORB ANNU”, “ORB FRAN”, “ORB FAVE”, “POR PORI”, “PSE STRI”, “PSE CLIV”, “SID SIDE”, “STE INTE”)

2. Which benthic categories are combined into “OTHER” in NCRMP_calculate_cover?

The following major benthic categories are currently classified as OTHER in the benthic cover metrics: SUBSTRATE, HYDROCORALS, OTHER INVERTEBRATES, CYANOBACTERIA, and SEAGRASS. Please see ncrmp_frrp_sppcodes, which is contained in the package, for information on how taxa are categorized into major benthic categories.

Disclaimer

This repository is a scientific product and is not official communication of the National Oceanic and Atmospheric Administration, or the United States Department of Commerce. All NOAA GitHub project code is provided on an ‘as is’ basis and the user assumes responsibility for its use. Any claims against the Department of Commerce or Department of Commerce bureaus stemming from the use of this GitHub project will be governed by all applicable Federal law. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by the Department of Commerce. The Department of Commerce seal and logo, or the seal and logo of a DOC bureau, shall not be used in any manner to imply endorsement of any commercial product or activity by DOC or the United States Government.