Cluster Administration Storage Tools

CAST stands for Cluster Administration Storage Tools.

CAST is comprised of several open source components:

CSM - Cluster System Management

A C API for managing a large cluster. Offers a suite of tools for maintaining the cluster:

• Discovery and management of system resources
• Database integration (PostgreSQL)
• Job launch support (workload management APIs)
• Node diagnostics (diag APIs and scripts)
• RAS events and actions
• Infrastructure Health checks
• Python Bindings for C APIs

Burst Buffer

A cost-effective mechanism that can improve I/O performance for a large class of high-performance computing applications without requirement of intermediary hardware. Burst Buffer provides:

• A fast storage tier between compute nodes and the traditional parallel file system
• Overlapping job stage-in and stage-out of data for checkpoint and restart
• Scratch volumes
• Extended memory I/O workloads

Function Shipping

A file I/O forwarding layer for Linux that aims to provide low-jitter access to remote parallel file system while retaining common POSIX semantics.

Table of Contents

• CSM APIs
  • Installation
  • Configuration
  • List of CSM APIs
  • Implementing New CSM APIs
  • CSM API Python Bindings Guide
  • Soft Failure Recovery
  • Change Log
• CSM Infrastructure
  • CSMD Executable
  • CSMD Configuration
• CSM Database
  • CSM Database Appendix
  • Using csm_db_history_archive.py
  • Using csm_db_backup_script_v1.sh
  • Using csm_db_connections_script.sh
  • Using csm_db_history_delete.py
  • Using csm_db_schema_version_upgrade_16_2.sh
  • Using csm_db_script.sh
  • Using csm_db_stats.sh script
  • Using csm_ras_type_script.sh
• Big Data Store
  • Elasticsearch
  • Kibana
  • Logstash
  • Common Big Data Store Problems
  • Data Aggregation
  • Beats
  • Python Guide
  • CSM Event Correlator Filter Plugin
  • Cast Search
• Tools
  • CSM logging tools