DOCUMENTATION

Introduction

• Main README
• In-depth Overview
• Getting Started
• CLI: overview
• CLI: reference guide
• Technical Blog
• How to deploy AIStore (useful links)

API and SDKs

• Go (language) API
• Python SDK, and also:
  • pip package
  • reference guide
• REST API
  • Easy URL
• Amazon S3
  • s3cmd client
  • S3 compatibility
  • Presigned S3 requests
  • Boto3 support

CLI

• Overview
• ais help
• Reference guide
• Monitoring
  • ais show cluster
  • ais show performance
  • ais show job
  • ais show config
• Cluster and node management
• Mountpath (disk) management
• Attach, detach, and monitor remote clusters
• Start, stop, and monitor downloads
• Distributed shuffle
• User account and access management
• Jobs

Storage Management

• Storage Services
• Buckets and Bucket Management
• CLI: Operations on Buckets
• CLI: ais show storage and subcommands
• CLI: ais storage and subcommands
• On-disk layout
• CLI: Three Ways to Evict Remote Bucket
• Backend Providers
• Virtual directories
• System files

Cluster Administration

• Node lifecycle: maintenance mode, rebalance/rebuild, shutdown, decommission
• CLI: ais cluster and subcommands
• Joining AIS cluster
• Leaving AIS cluster
• Global Rebalance
• Highly available control plane
• Start/stop maintenance mode, shutdown, decommission, and related operations
• Out-of-band updates
• Troubleshooting

Configuration and Security

• Configuration
• Environment variables
• CLI: ais config
• Feature flags
• Security and Access Control
  • Authentication Server (AuthN)
• HTTPS: loading, reloading, and generating certificates; switching cluster between HTTP and HTTPS
  • Managing TLS Certificates

ETL - Extract, Transform, Load

• AIStore ETL Overview
• ETL CLI Docs
• ETL Python SDK Examples
• Custom Transformers
• ETL Python Webserver SDK
• ETL Golang Webserver Package

Advanced Features and Tools

• Reading, writing, and listing archives
• Distributed Shuffle (dsort)
• Initial Sharding utility (ishard)
• Downloader
• Tools and utilities
• Batch operations
• eXtended Actions (xactions)
• CLI: ais job and ais show job, including:
  • prefetch remote datasets
  • copy (list, range, and/or prefix) selected objects or entire (in-cluster or remote) buckets
  • download remote BLOBs
  • promote NFS or SMB share

Performance and Resilience

• AIS Load Generator: integrated benchmark tool
• How to benchmark
• Performance tuning and testing
• Performance monitoring
• Using Rate Limits to Enhance Reliability and Performance
• Checksumming: brief theory of operations
• Blog: Maintaining Resilient Connectivity During Lifecycle Events

Observability and Monitoring

• Introduction to AIS observability
• Configuring, accessing, and utilizing AIS logs
• Prometheus integration
• Metrics reference
• Visualizing AIS metrics with Grafana
• Working with Kubernetes monitoring stacks

Developer Resources

• Getting Started
• How to deploy AIStore (useful links)
• Docker
• Assorted scripts; profiling
• aisnode command line

Production Deployment

• AIStore on Kubernetes
• Kubernetes Operator
• Ansible Playbooks
• Helm Charts
• Monitoring

Special Cases and Advanced Topics

• Unicode and Special Symbols in Object Names
• Extremely Long Object Names
• Blog: Split-brain is Inevitable
• Blog: Comparing OCI’s Native Object Storage and S3 API Backends
• Blog: AIStore Python SDK: Maintaining Resilient Connectivity During Lifecycle Events
• Blog: Adding Data to AIStore -- PUT Performance
• Traffic patterns