OP#792 initial commit after fork
This commit is contained in:
91
docs/distributed/CONFIG.md
Normal file
91
docs/distributed/CONFIG.md
Normal file
@@ -0,0 +1,91 @@
|
||||
## MinIO configuration YAML
|
||||
|
||||
MinIO now supports starting the server arguments and configuration via a YAML configuration file. This YAML configuration describes everything that can be configured in a MinIO setup, such as '--address', '--console-address' and command line arguments for the MinIO server.
|
||||
|
||||
Historically everything to MinIO was provided via command arguments for the hostnames and the drives via an ellipses syntax such as `minio server http://host{1...4}/disk{1...4}` this requirement added an additional burden to have sequential hostnames for us to make sure that we can provide horizontal distribution, however we have come across situations where sometimes this is not feasible and there are no easier alternatives without modifying /etc/hosts on the host system as root user. Many times in airgapped deployments this is not allowed or requires audits and approvals.
|
||||
|
||||
MinIO server configuration file allows users to provide topology that allows for heterogeneous hostnames, allowing MinIO to deployed in pre-existing environments without any further OS level configurations.
|
||||
|
||||
### Usage
|
||||
|
||||
```
|
||||
minio server --config config.yaml
|
||||
```
|
||||
|
||||
Lets you start MinIO server with all inputs to start MinIO server provided via this configuration file, once the configuration file is provided all other pre-existing values on disk for configuration are overridden by the new values set in this configuration file.
|
||||
|
||||
Following is an example YAML configuration structure.
|
||||
```yaml
|
||||
version: v2
|
||||
address: ":9000"
|
||||
rootUser: "minioadmin"
|
||||
rootPassword: "minioadmin"
|
||||
console-address: ":9001"
|
||||
certs-dir: "/home/user/.minio/certs/"
|
||||
pools: # Specify the nodes and drives with pools
|
||||
- args:
|
||||
- "https://server-example-pool1:9000/mnt/disk{1...4}/"
|
||||
- "https://server{1...2}-pool1:9000/mnt/disk{1...4}/"
|
||||
- "https://server3-pool1:9000/mnt/disk{1...4}/"
|
||||
- "https://server4-pool1:9000/mnt/disk{1...4}/"
|
||||
- args:
|
||||
- "https://server-example-pool2:9000/mnt/disk{1...4}/"
|
||||
- "https://server{1...2}-pool2:9000/mnt/disk{1...4}/"
|
||||
- "https://server3-pool2:9000/mnt/disk{1...4}/"
|
||||
- "https://server4-pool2:9000/mnt/disk{1...4}/"
|
||||
# more args
|
||||
|
||||
options:
|
||||
ftp: # settings for MinIO to act as an ftp server
|
||||
address: ":8021"
|
||||
passive-port-range: "30000-40000"
|
||||
sftp: # settings for MinIO to act as an sftp server
|
||||
address: ":8022"
|
||||
ssh-private-key: "/home/user/.ssh/id_rsa"
|
||||
```
|
||||
|
||||
If you are using the config `v1` YAML you should migrate your `pools:` field values to the following format
|
||||
|
||||
`v1` format
|
||||
```yaml
|
||||
pools: # Specify the nodes and drives with pools
|
||||
-
|
||||
- "https://server-example-pool1:9000/mnt/disk{1...4}/"
|
||||
- "https://server{1...2}-pool1:9000/mnt/disk{1...4}/"
|
||||
- "https://server3-pool1:9000/mnt/disk{1...4}/"
|
||||
- "https://server4-pool1:9000/mnt/disk{1...4}/"
|
||||
```
|
||||
|
||||
to `v2` format
|
||||
|
||||
```yaml
|
||||
pools:
|
||||
- args:
|
||||
- "https://server-example-pool1:9000/mnt/disk{1...4}/"
|
||||
- "https://server{1...2}-pool1:9000/mnt/disk{1...4}/"
|
||||
- "https://server3-pool1:9000/mnt/disk{1...4}/"
|
||||
- "https://server4-pool1:9000/mnt/disk{1...4}/"
|
||||
set-drive-count: 4 # Advanced option, must be used under guidance from MinIO team.
|
||||
```
|
||||
|
||||
### Things to know
|
||||
|
||||
- Fields such as `version` and `pools` are mandatory, however all other fields are optional.
|
||||
- Each pool expects a minimum of 2 nodes per pool, and unique non-repeating hosts for each argument.
|
||||
- Each pool expects each host in this pool has the same number of drives specified as any other host.
|
||||
- Mixing `local-path` and `distributed-path` is not allowed, doing so would cause MinIO to refuse starting the server.
|
||||
- Ellipses and bracket notation (e.g. `{1...10}`) are allowed.
|
||||
|
||||
> NOTE: MinIO environmental variables still take precedence over the `config.yaml` file, however `config.yaml` is preferred over MinIO internal config KV settings via `mc admin config set alias/ <sub-system>`.
|
||||
|
||||
### TODO
|
||||
|
||||
In subsequent releases we are planning to extend this to provide things like
|
||||
|
||||
- Reload() of MinIO server arguments without fully restarting the process.
|
||||
|
||||
- Expanding 1 node at a time by automating the process of creating a new pool
|
||||
and decommissioning to provide a functionality that smaller deployments
|
||||
care about.
|
||||
|
||||
- Fully allow bracket notation (e.g. `{a,c,f}`) to have multiple entries on one line.
|
||||
122
docs/distributed/DECOMMISSION.md
Normal file
122
docs/distributed/DECOMMISSION.md
Normal file
@@ -0,0 +1,122 @@
|
||||
# Decommissioning
|
||||
|
||||
Decommissiong is a mechanism in MinIO to drain older pools (usually with old hardware) and migrate the content from such pools to a newer pools (usually better hardware). Decommissioning spreads the data across all pools - for example, if you decommission `pool1`, all the data from `pool1` spreads across `pool2` and `pool3`.
|
||||
|
||||
## Features
|
||||
|
||||
- A pool in decommission still allows READ access to all its contents, newer WRITEs will automatically be scheduled to only pools not in decommission status.
|
||||
- All versioned buckets maintain the same order for "versions" for each object after being decommissioned to the other pools.
|
||||
- A pool interrupted during the decommission process, such as for a cluster restart, resumes from where it left off.
|
||||
|
||||
## How to decommission a pool
|
||||
|
||||
```
|
||||
λ mc admin decommission start alias/ http://minio{1...2}/data{1...4}
|
||||
```
|
||||
|
||||
## Status decommissioning a pool
|
||||
|
||||
### Decommissioning without args lists all pools
|
||||
|
||||
```
|
||||
λ mc admin decommission status alias/
|
||||
┌─────┬─────────────────────────────────┬──────────────────────────────────┬────────┐
|
||||
│ ID │ Pools │ Capacity │ Status │
|
||||
│ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Active │
|
||||
│ 2nd │ http://minio{3...4}/data{1...4} │ 329 GiB (used) / 421 GiB (total) │ Active │
|
||||
└─────┴─────────────────────────────────┴──────────────────────────────────┴────────┘
|
||||
```
|
||||
|
||||
### Decommissioning status
|
||||
|
||||
```
|
||||
λ mc admin decommission status alias/ http://minio{1...2}/data{1...4}
|
||||
Decommissioning rate at 36 MiB/sec [4 TiB/50 TiB]
|
||||
Started: 1 minute ago
|
||||
```
|
||||
|
||||
Once it is **Complete**
|
||||
|
||||
```
|
||||
λ mc admin decommission status alias/ http://minio{1...2}/data{1...4}
|
||||
Decommission of pool http://minio{1...2}/data{1...4} is complete, you may now remove it from server command line
|
||||
```
|
||||
|
||||
### A pool not under decommissioning will throw an error
|
||||
|
||||
```
|
||||
λ mc admin decommission status alias/ http://minio{1...2}/data{1...4}
|
||||
ERROR: This pool is not scheduled for decommissioning currently.
|
||||
```
|
||||
|
||||
## Canceling a decommission
|
||||
|
||||
Stop an on-going decommission in progress, mainly used in situations when the load may be too high and you may want to schedule the decommission at a later point in time.
|
||||
|
||||
`mc admin decommission cancel` without an argument, lists out any on-going decommission in progress.
|
||||
|
||||
```
|
||||
λ mc admin decommission cancel alias/
|
||||
┌─────┬─────────────────────────────────┬──────────────────────────────────┬──────────┐
|
||||
│ ID │ Pools │ Capacity │ Status │
|
||||
│ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Draining │
|
||||
└─────┴─────────────────────────────────┴──────────────────────────────────┴──────────┘
|
||||
```
|
||||
|
||||
> NOTE: Canceled decommission will not make the pool active again, since we might have potentially partial namespace on the other pools, to avoid this scenario be absolutely sure to make decommissioning a planned well thought activity. This is not to be run on a daily basis.
|
||||
|
||||
```
|
||||
λ mc admin decommission cancel alias/ http://minio{1...2}/data{1...4}
|
||||
┌─────┬─────────────────────────────────┬──────────────────────────────────┬────────────────────┐
|
||||
│ ID │ Pools │ Capacity │ Status │
|
||||
│ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Draining(Canceled) │
|
||||
└─────┴─────────────────────────────────┴──────────────────────────────────┴────────────────────┘
|
||||
```
|
||||
|
||||
If the decommission process fails for any reason, the status indicates failed.
|
||||
|
||||
```
|
||||
λ mc admin decommission status alias/
|
||||
┌─────┬─────────────────────────────────┬──────────────────────────────────┬──────────────────┐
|
||||
│ ID │ Pools │ Capacity │ Status │
|
||||
│ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Draining(Failed) │
|
||||
│ 2nd │ http://minio{3...4}/data{1...4} │ 329 GiB (used) / 421 GiB (total) │ Active │
|
||||
└─────┴─────────────────────────────────┴──────────────────────────────────┴──────────────────┘
|
||||
```
|
||||
|
||||
## Restart a canceled or failed decommission
|
||||
|
||||
```
|
||||
λ mc admin decommission start alias/ http://minio{1...2}/data{1...4}
|
||||
```
|
||||
|
||||
## When decommission is 'Complete'
|
||||
|
||||
Once decommission is complete, it will be indicated with *Complete* status. *Complete* means that now you can now safely remove the first pool argument from the MinIO command line.
|
||||
|
||||
```
|
||||
λ mc admin decommission status alias/
|
||||
┌─────┬─────────────────────────────────┬──────────────────────────────────┬──────────┐
|
||||
│ ID │ Pools │ Capacity │ Status │
|
||||
│ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Complete │
|
||||
│ 2nd │ http://minio{3...4}/data{1...4} │ 329 GiB (used) / 421 GiB (total) │ Active │
|
||||
└─────┴─────────────────────────────────┴──────────────────────────────────┴──────────┘
|
||||
```
|
||||
|
||||
- On baremetal setups, if you have `MINIO_VOLUMES="http://minio{1...2}/data{1...4} http://minio{3...4}/data{1...4}"`, you can remove the first argument `http://minio{1...2}/data{1...4}` to update your `MINIO_VOLUMES` setting, then restart all the servers in the setup in parallel using `systemctl restart minio`.
|
||||
|
||||
- On Kubernetes setups, the statefulset specification needs to be modified by changing the command line input for the MinIO container. Once the relevant changes are done, proceed to execute `kubectl apply -f statefulset.yaml`.
|
||||
|
||||
- On Operator based MinIO deployments, you need to modify the `tenant.yaml` specification and modify the `pools:` section from two entries to a single entry. After making relevant changes, proceed to execute `kubectl apply -f tenant.yaml`.
|
||||
|
||||
> Without a 'Complete' status any 'Active' or 'Draining' pool(s) are not allowed to be removed once configured.
|
||||
|
||||
## NOTE
|
||||
|
||||
- Empty delete markers (such as for objects with no other successor versions) do not transition to the new pool to avoid creating empty metadata on the other pool(s). If you believe transitioning empty delete markers is required, open a GitHub issue.
|
||||
|
||||
## TODO
|
||||
|
||||
- Richer progress UI is not present at the moment, this will be addressed in subsequent releases. Currently however a RATE of data transfer and usage increase is displayed via `mc`.
|
||||
- Transitioned Hot Tier's as pooled setups are not currently supported, attempting to decommission buckets with ILM Transition will be rejected by the server. This will be supported in future releases.
|
||||
- Embedded Console UI does not support Decommissioning through the UI yet. This will be supported in future releases.
|
||||
166
docs/distributed/DESIGN.md
Normal file
166
docs/distributed/DESIGN.md
Normal file
@@ -0,0 +1,166 @@
|
||||
# Distributed Server Design Guide [](https://slack.min.io)
|
||||
|
||||
This document explains the design, architecture and advanced use cases of the MinIO distributed server.
|
||||
|
||||
## Command-line
|
||||
|
||||
```
|
||||
NAME:
|
||||
minio server - start object storage server
|
||||
|
||||
USAGE:
|
||||
minio server [FLAGS] DIR1 [DIR2..]
|
||||
minio server [FLAGS] DIR{1...64}
|
||||
minio server [FLAGS] DIR{1...64} DIR{65...128}
|
||||
|
||||
DIR:
|
||||
DIR points to a directory on a filesystem. When you want to combine
|
||||
multiple drives into a single large system, pass one directory per
|
||||
filesystem separated by space. You may also use a '...' convention
|
||||
to abbreviate the directory arguments. Remote directories in a
|
||||
distributed setup are encoded as HTTP(s) URIs.
|
||||
```
|
||||
|
||||
## Common usage
|
||||
|
||||
Standalone erasure coded configuration with 4 sets with 16 drives each.
|
||||
|
||||
```
|
||||
minio server dir{1...64}
|
||||
```
|
||||
|
||||
Distributed erasure coded configuration with 64 sets with 16 drives each.
|
||||
|
||||
```
|
||||
minio server http://host{1...16}/export{1...64}
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
Expansion of ellipses and choice of erasure sets based on this expansion is an automated process in MinIO. Here are some of the details of our underlying erasure coding behavior.
|
||||
|
||||
- Erasure coding used by MinIO is [Reed-Solomon](https://github.com/klauspost/reedsolomon) erasure coding scheme, which has a total shard maximum of 256 i.e 128 data and 128 parity. MinIO design goes beyond this limitation by doing some practical architecture choices.
|
||||
|
||||
- Erasure set is a single erasure coding unit within a MinIO deployment. An object is sharded within an erasure set. Erasure set size is automatically calculated based on the number of drives. MinIO supports unlimited number of drives but each erasure set can be up to 16 drives and a minimum of 2 drives.
|
||||
|
||||
- We limited the number of drives to 16 for erasure set because, erasure code shards more than 16 can become chatty and do not have any performance advantages. Additionally since 16 drive erasure set gives you tolerance of 8 drives per object by default which is plenty in any practical scenario.
|
||||
|
||||
- Choice of erasure set size is automatic based on the number of drives available, let's say for example if there are 32 servers and 32 drives which is a total of 1024 drives. In this scenario 16 becomes the erasure set size. This is decided based on the greatest common divisor (GCD) of acceptable erasure set sizes ranging from *4 to 16*.
|
||||
|
||||
- *If total drives has many common divisors the algorithm chooses the minimum amounts of erasure sets possible for a erasure set size of any N*. In the example with 1024 drives - 4, 8, 16 are GCD factors. With 16 drives we get a total of 64 possible sets, with 8 drives we get a total of 128 possible sets, with 4 drives we get a total of 256 possible sets. So algorithm automatically chooses 64 sets, which is *16* 64 = 1024* drives in total.
|
||||
|
||||
- *If total number of nodes are of odd number then GCD algorithm provides affinity towards odd number erasure sets to provide for uniform distribution across nodes*. This is to ensure that same number of drives are pariticipating in any erasure set. For example if you have 2 nodes with 180 drives then GCD is 15 but this would lead to uneven distribution, one of the nodes would participate more drives. To avoid this the affinity is given towards nodes which leads to next best GCD factor of 12 which provides uniform distribution.
|
||||
|
||||
- In this algorithm, we also make sure that we spread the drives out evenly. MinIO server expands ellipses passed as arguments. Here is a sample expansion to demonstrate the process.
|
||||
|
||||
```
|
||||
minio server http://host{1...2}/export{1...8}
|
||||
```
|
||||
|
||||
Expected expansion
|
||||
|
||||
```
|
||||
> http://host1/export1
|
||||
> http://host2/export1
|
||||
> http://host1/export2
|
||||
> http://host2/export2
|
||||
> http://host1/export3
|
||||
> http://host2/export3
|
||||
> http://host1/export4
|
||||
> http://host2/export4
|
||||
> http://host1/export5
|
||||
> http://host2/export5
|
||||
> http://host1/export6
|
||||
> http://host2/export6
|
||||
> http://host1/export7
|
||||
> http://host2/export7
|
||||
> http://host1/export8
|
||||
> http://host2/export8
|
||||
```
|
||||
|
||||
*A noticeable trait of this expansion is that it chooses unique hosts such the setup provides maximum protection and availability.*
|
||||
|
||||
- Choosing an erasure set for the object is decided during `PutObject()`, object names are used to find the right erasure set using the following pseudo code.
|
||||
|
||||
```go
|
||||
// hashes the key returning an integer.
|
||||
func sipHashMod(key string, cardinality int, id [16]byte) int {
|
||||
if cardinality <= 0 {
|
||||
return -1
|
||||
}
|
||||
sip := siphash.New(id[:])
|
||||
sip.Write([]byte(key))
|
||||
return int(sip.Sum64() % uint64(cardinality))
|
||||
}
|
||||
```
|
||||
|
||||
Input for the key is the object name specified in `PutObject()`, returns a unique index. This index is one of the erasure sets where the object will reside. This function is a consistent hash for a given object name i.e for a given object name the index returned is always the same.
|
||||
|
||||
- Write and Read quorum are required to be satisfied only across the erasure set for an object. Healing is also done per object within the erasure set which contains the object.
|
||||
|
||||
- MinIO does erasure coding at the object level not at the volume level, unlike other object storage vendors. This allows applications to choose different storage class by setting `x-amz-storage-class=STANDARD/REDUCED_REDUNDANCY` for each object uploads so effectively utilizing the capacity of the cluster. Additionally these can also be enforced using IAM policies to make sure the client uploads with correct HTTP headers.
|
||||
|
||||
- MinIO also supports expansion of existing clusters in server pools. Each pool is a self contained entity with same SLA's (read/write quorum) for each object as original cluster. By using the existing namespace for lookup validation MinIO ensures conflicting objects are not created. When no such object exists then MinIO simply uses the least used pool to place new objects.
|
||||
|
||||
### There are no limits on how many server pools can be combined
|
||||
|
||||
```
|
||||
minio server http://host{1...32}/export{1...32} http://host{1...12}/export{1...12}
|
||||
```
|
||||
|
||||
In above example there are two server pools
|
||||
|
||||
- 32 * 32 = 1024 drives pool1
|
||||
- 12 * 12 = 144 drives pool2
|
||||
|
||||
> Notice the requirement of common SLA here original cluster had 1024 drives with 16 drives per erasure set with default parity of '4', second pool is expected to have a minimum of 8 drives per erasure set to match the original cluster SLA (parity count) of '4'. '12' drives stripe per erasure set in the second pool satisfies the original pool's parity count.
|
||||
|
||||
Refer to the sizing guide with details on the default parity count chosen for different erasure stripe sizes [here](https://github.com/minio/minio/blob/master/docs/distributed/SIZING.md)
|
||||
|
||||
MinIO places new objects in server pools based on proportionate free space, per pool. Following pseudo code demonstrates this behavior.
|
||||
|
||||
```go
|
||||
func getAvailablePoolIdx(ctx context.Context) int {
|
||||
serverPools := z.getServerPoolsAvailableSpace(ctx)
|
||||
total := serverPools.TotalAvailable()
|
||||
// choose when we reach this many
|
||||
choose := rand.Uint64() % total
|
||||
atTotal := uint64(0)
|
||||
for _, pool := range serverPools {
|
||||
atTotal += pool.Available
|
||||
if atTotal > choose && pool.Available > 0 {
|
||||
return pool.Index
|
||||
}
|
||||
}
|
||||
// Should not happen, but print values just in case.
|
||||
panic(fmt.Errorf("reached end of serverPools (total: %v, atTotal: %v, choose: %v)", total, atTotal, choose))
|
||||
}
|
||||
```
|
||||
|
||||
## Other usages
|
||||
|
||||
### Advanced use cases with multiple ellipses
|
||||
|
||||
Standalone erasure coded configuration with 4 sets with 16 drives each, which spawns drives across controllers.
|
||||
|
||||
```
|
||||
minio server /mnt/controller{1...4}/data{1...16}
|
||||
```
|
||||
|
||||
Standalone erasure coded configuration with 16 sets, 16 drives per set, across mounts and controllers.
|
||||
|
||||
```
|
||||
minio server /mnt{1...4}/controller{1...4}/data{1...16}
|
||||
```
|
||||
|
||||
Distributed erasure coded configuration with 2 sets, 16 drives per set across hosts.
|
||||
|
||||
```
|
||||
minio server http://host{1...32}/disk1
|
||||
```
|
||||
|
||||
Distributed erasure coded configuration with rack level redundancy 32 sets in total, 16 drives per set.
|
||||
|
||||
```
|
||||
minio server http://rack{1...4}-host{1...8}.example.net/export{1...16}
|
||||
```
|
||||
109
docs/distributed/README.md
Normal file
109
docs/distributed/README.md
Normal file
@@ -0,0 +1,109 @@
|
||||
# Distributed MinIO Quickstart Guide [](https://slack.min.io) [](https://hub.docker.com/r/minio/minio/)
|
||||
|
||||
MinIO in distributed mode lets you pool multiple drives (even on different machines) into a single object storage server. As drives are distributed across several nodes, distributed MinIO can withstand multiple node failures and yet ensure full data protection.
|
||||
|
||||
## Why distributed MinIO?
|
||||
|
||||
MinIO in distributed mode can help you setup a highly-available storage system with a single object storage deployment. With distributed MinIO, you can optimally use storage devices, irrespective of their location in a network.
|
||||
|
||||
### Data protection
|
||||
|
||||
Distributed MinIO provides protection against multiple node/drive failures and [bit rot](https://github.com/minio/minio/blob/master/docs/erasure/README.md#what-is-bit-rot-protection) using [erasure code](https://docs.min.io/community/minio-object-store/operations/concepts/erasure-coding.html). As the minimum drives required for distributed MinIO is 2 (same as minimum drives required for erasure coding), erasure code automatically kicks in as you launch distributed MinIO.
|
||||
|
||||
If one or more drives are offline at the start of a PutObject or NewMultipartUpload operation the object will have additional data protection bits added automatically to provide additional safety for these objects.
|
||||
|
||||
### High availability
|
||||
|
||||
A stand-alone MinIO server would go down if the server hosting the drives goes offline. In contrast, a distributed MinIO setup with _m_ servers and _n_ drives will have your data safe as long as _m/2_ servers or _m*n_/2 or more drives are online.
|
||||
|
||||
For example, an 16-server distributed setup with 200 drives per node would continue serving files, up to 4 servers can be offline in default configuration i.e around 800 drives down MinIO would continue to read and write objects.
|
||||
|
||||
Refer to sizing guide for more understanding on default values chosen depending on your erasure stripe size [here](https://github.com/minio/minio/blob/master/docs/distributed/SIZING.md). Parity settings can be changed using [storage classes](https://github.com/minio/minio/tree/master/docs/erasure/storage-class).
|
||||
|
||||
### Consistency Guarantees
|
||||
|
||||
MinIO follows strict **read-after-write** and **list-after-write** consistency model for all i/o operations both in distributed and standalone modes. This consistency model is only guaranteed if you use disk filesystems such as xfs, zfs or btrfs etc.. for distributed setup.
|
||||
|
||||
**In our tests we also found ext4 does not honor POSIX O_DIRECT/Fdatasync semantics, ext4 trades performance for consistency guarantees. Please avoid ext4 in your setup.**
|
||||
|
||||
**If MinIO distributed setup is using NFS volumes underneath it is not guaranteed MinIO will provide these consistency guarantees since NFS is not strictly consistent (If you must use NFS we recommend that you at least use NFSv4 instead of NFSv3 for relatively better outcomes).**
|
||||
|
||||
## Get started
|
||||
|
||||
If you're aware of stand-alone MinIO set up, the process remains largely the same. MinIO server automatically switches to stand-alone or distributed mode, depending on the command line parameters.
|
||||
|
||||
### 1. Prerequisites
|
||||
|
||||
Install MinIO either on Kubernetes or Distributed Linux.
|
||||
|
||||
Install MinIO on Kubernetes:
|
||||
|
||||
- [MinIO Quickstart Guide for Kubernetes](https://docs.min.io/community/minio-object-store/operations/deployments/kubernetes.html).
|
||||
- [Deploy a Tenant from the MinIO Operator](https://docs.min.io/community/minio-object-store/operations/deployments/k8s-deploy-minio-tenant-on-kubernetes.html)
|
||||
|
||||
Install Distributed MinIO on Linux:
|
||||
- [Deploy Distributed MinIO on Linux](https://docs.min.io/community/minio-object-store/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html#deploy-distributed-minio)
|
||||
|
||||
### 2. Run distributed MinIO
|
||||
|
||||
To start a distributed MinIO instance, you just need to pass drive locations as parameters to the minio server command. Then, you’ll need to run the same command on all the participating nodes.
|
||||
|
||||
**NOTE:**
|
||||
|
||||
- All the nodes running distributed MinIO should share a common root credentials, for the nodes to connect and trust each other. To achieve this, it is **recommended** to export root user and root password as environment variables, `MINIO_ROOT_USER` and `MINIO_ROOT_PASSWORD`, on all the nodes before executing MinIO server command. If not exported, default `minioadmin/minioadmin` credentials shall be used.
|
||||
- **MinIO creates erasure-coding sets of _2_ to _16_ drives per set. The number of drives you provide in total must be a multiple of one of those numbers.**
|
||||
- **MinIO chooses the largest EC set size which divides into the total number of drives or total number of nodes given - making sure to keep the uniform distribution i.e each node participates equal number of drives per set**.
|
||||
- **Each object is written to a single EC set, and therefore is spread over no more than 16 drives.**
|
||||
- **All the nodes running distributed MinIO setup are recommended to be homogeneous, i.e. same operating system, same number of drives and same network interconnects.**
|
||||
- MinIO distributed mode requires **fresh directories**. If required, the drives can be shared with other applications. You can do this by using a sub-directory exclusive to MinIO. For example, if you have mounted your volume under `/export`, pass `/export/data` as arguments to MinIO server.
|
||||
- The IP addresses and drive paths below are for demonstration purposes only, you need to replace these with the actual IP addresses and drive paths/folders.
|
||||
- Servers running distributed MinIO instances should be less than 15 minutes apart. You can enable [NTP](http://www.ntp.org/) service as a best practice to ensure same times across servers.
|
||||
- `MINIO_DOMAIN` environment variable should be defined and exported for bucket DNS style support.
|
||||
- Running Distributed MinIO on **Windows** operating system is considered **experimental**. Please proceed with caution.
|
||||
|
||||
Example 1: Start distributed MinIO instance on n nodes with m drives each mounted at `/export1` to `/exportm` (pictured below), by running this command on all the n nodes:
|
||||
|
||||
|
||||
|
||||
### GNU/Linux and macOS
|
||||
|
||||
```sh
|
||||
export MINIO_ROOT_USER=<ACCESS_KEY>
|
||||
export MINIO_ROOT_PASSWORD=<SECRET_KEY>
|
||||
minio server http://host{1...n}/export{1...m}
|
||||
```
|
||||
|
||||
> **NOTE:** In above example `n` and `m` represent positive integers, _do not copy paste and expect it work make the changes according to local deployment and setup_.
|
||||
> **NOTE:** `{1...n}` shown have 3 dots! Using only 2 dots `{1..n}` will be interpreted by your shell and won't be passed to MinIO server, affecting the erasure coding order, which would impact performance and high availability. **Always use ellipses syntax `{1...n}` (3 dots!) for optimal erasure-code distribution**
|
||||
|
||||
### Expanding existing distributed setup
|
||||
|
||||
MinIO supports expanding distributed erasure coded clusters by specifying new set of clusters on the command-line as shown below:
|
||||
|
||||
```sh
|
||||
export MINIO_ROOT_USER=<ACCESS_KEY>
|
||||
export MINIO_ROOT_PASSWORD=<SECRET_KEY>
|
||||
minio server http://host{1...n}/export{1...m} http://host{o...z}/export{1...m}
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
```
|
||||
minio server http://host{1...4}/export{1...16} http://host{5...12}/export{1...16}
|
||||
```
|
||||
|
||||
Now the server has expanded total storage by _(newly_added_servers\*m)_ more drives, taking the total count to _(existing_servers\*m)+(newly_added_servers\*m)_ drives. New object upload requests automatically start using the least used cluster. This expansion strategy works endlessly, so you can perpetually expand your clusters as needed. When you restart, it is immediate and non-disruptive to the applications. Each group of servers in the command-line is called a pool. There are 2 server pools in this example. New objects are placed in server pools in proportion to the amount of free space in each pool. Within each pool, the location of the erasure-set of drives is determined based on a deterministic hashing algorithm.
|
||||
|
||||
> **NOTE:** **Each pool you add must have the same erasure coding parity configuration as the original pool, so the same data redundancy SLA is maintained.**
|
||||
|
||||
## 3. Test your setup
|
||||
|
||||
To test this setup, access the MinIO server via browser or [`mc`](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart).
|
||||
|
||||
## Explore Further
|
||||
|
||||
- [MinIO Erasure Code QuickStart Guide](https://docs.min.io/community/minio-object-store/operations/concepts/erasure-coding.html)
|
||||
- [Use `mc` with MinIO Server](https://docs.min.io/community/minio-object-store/reference/minio-mc.html)
|
||||
- [Use `aws-cli` with MinIO Server](https://docs.min.io/community/minio-object-store/integrations/aws-cli-with-minio.html)
|
||||
- [Use `minio-go` SDK with MinIO Server](https://docs.min.io/community/minio-object-store/developers/go/minio-go.html)
|
||||
- [The MinIO documentation website](https://docs.min.io/community/minio-object-store/index.html)
|
||||
39
docs/distributed/SIZING.md
Normal file
39
docs/distributed/SIZING.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# Erasure code sizing guide
|
||||
|
||||
## Toy Setups
|
||||
|
||||
Capacity constrained environments, MinIO will work but not recommended for production.
|
||||
|
||||
| servers | drives (per node) | stripe_size | parity chosen (default) | tolerance for reads (servers) | tolerance for writes (servers) |
|
||||
|--------:|------------------:|------------:|------------------------:|------------------------------:|-------------------------------:|
|
||||
| 1 | 1 | 1 | 0 | 0 | 0 |
|
||||
| 1 | 4 | 4 | 2 | 0 | 0 |
|
||||
| 4 | 1 | 4 | 2 | 2 | 1 |
|
||||
| 5 | 1 | 5 | 2 | 2 | 2 |
|
||||
| 6 | 1 | 6 | 3 | 3 | 2 |
|
||||
| 7 | 1 | 7 | 3 | 3 | 3 |
|
||||
|
||||
## Minimum System Configuration for Production
|
||||
|
||||
| servers | drives (per node) | stripe_size | parity chosen (default) | tolerance for reads (servers) | tolerance for writes (servers) |
|
||||
|--------:|------------------:|------------:|------------------------:|------------------------------:|-------------------------------:|
|
||||
| 4 | 2 | 8 | 4 | 2 | 1 |
|
||||
| 5 | 2 | 10 | 4 | 2 | 2 |
|
||||
| 6 | 2 | 12 | 4 | 2 | 2 |
|
||||
| 7 | 2 | 14 | 4 | 2 | 2 |
|
||||
| 8 | 1 | 8 | 4 | 4 | 3 |
|
||||
| 8 | 2 | 16 | 4 | 2 | 2 |
|
||||
| 9 | 2 | 9 | 4 | 4 | 4 |
|
||||
| 10 | 2 | 10 | 4 | 4 | 4 |
|
||||
| 11 | 2 | 11 | 4 | 4 | 4 |
|
||||
| 12 | 2 | 12 | 4 | 4 | 4 |
|
||||
| 13 | 2 | 13 | 4 | 4 | 4 |
|
||||
| 14 | 2 | 14 | 4 | 4 | 4 |
|
||||
| 15 | 2 | 15 | 4 | 4 | 4 |
|
||||
| 16 | 2 | 16 | 4 | 4 | 4 |
|
||||
|
||||
If one or more drives are offline at the start of a PutObject or NewMultipartUpload operation the object will have additional data
|
||||
protection bits added automatically to provide the regular safety for these objects up to 50% of the number of drives.
|
||||
This will allow normal write operations to take place on systems that exceed the write tolerance.
|
||||
|
||||
This means that in the examples above the system will always write 4 parity shards at the expense of slightly higher disk usage.
|
||||
153
docs/distributed/decom-compressed-sse-s3.sh
Executable file
153
docs/distributed/decom-compressed-sse-s3.sh
Executable file
@@ -0,0 +1,153 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ -n "$TEST_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
|
||||
pkill minio
|
||||
rm -rf /tmp/xl
|
||||
|
||||
if [ ! -f ./mc ]; then
|
||||
wget --quiet -O mc https://dl.minio.io/client/mc/release/linux-amd64/mc &&
|
||||
chmod +x mc
|
||||
fi
|
||||
|
||||
export CI=true
|
||||
export MINIO_COMPRESSION_ENABLE="on"
|
||||
export MINIO_COMPRESSION_EXTENSIONS=".go"
|
||||
export MINIO_COMPRESSION_MIME_TYPES="application/*"
|
||||
export MINIO_COMPRESSION_ALLOW_ENCRYPTION="on"
|
||||
export MINIO_KMS_AUTO_ENCRYPTION=on
|
||||
export MINIO_KMS_SECRET_KEY=my-minio-key:OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw=
|
||||
export MC_HOST_myminio="http://minioadmin:minioadmin@localhost:9000/"
|
||||
|
||||
(minio server http://localhost:9000/tmp/xl/{1...10}/disk{0...1} 2>&1 >/dev/null) &
|
||||
pid=$!
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
./mc admin user add myminio/ minio123 minio123
|
||||
./mc admin user add myminio/ minio12345 minio12345
|
||||
|
||||
./mc admin policy create myminio/ rw ./docs/distributed/rw.json
|
||||
./mc admin policy create myminio/ lake ./docs/distributed/rw.json
|
||||
|
||||
./mc admin policy attach myminio/ rw --user=minio123
|
||||
./mc admin policy attach myminio/ lake --user=minio12345
|
||||
|
||||
./mc mb -l myminio/versioned
|
||||
|
||||
./mc mirror internal myminio/versioned/ --quiet >/dev/null
|
||||
|
||||
## Soft delete (creates delete markers)
|
||||
./mc rm -r --force myminio/versioned >/dev/null
|
||||
|
||||
## mirror again to create another set of version on top
|
||||
./mc mirror internal myminio/versioned/ --quiet >/dev/null
|
||||
|
||||
expected_checksum=$(./mc cat internal/dsync/drwmutex.go | md5sum)
|
||||
|
||||
user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
kill $pid
|
||||
|
||||
(minio server http://localhost:9000/tmp/xl/{1...10}/disk{0...1} http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/expanded_1.log) &
|
||||
pid_1=$!
|
||||
|
||||
(minio server --address ":9001" http://localhost:9000/tmp/xl/{1...10}/disk{0...1} http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/expanded_2.log) &
|
||||
pid_2=$!
|
||||
|
||||
sleep 30
|
||||
./mc ready myminio
|
||||
|
||||
expanded_user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
expanded_policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
if [ $user_count -ne $expanded_user_count ]; then
|
||||
echo "BUG: original user count differs from expanded setup"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ $policy_count -ne $expanded_policy_count ]; then
|
||||
echo "BUG: original policy count differs from expanded setup"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc version info myminio/versioned | grep -q "versioning is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "expected versioning enabled after expansion"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc mirror cmd myminio/versioned/ --quiet >/dev/null
|
||||
|
||||
./mc ls -r myminio/versioned/ >expanded_ns.txt
|
||||
./mc ls -r --versions myminio/versioned/ >expanded_ns_versions.txt
|
||||
|
||||
./mc admin decom start myminio/ http://localhost:9000/tmp/xl/{1...10}/disk{0...1}
|
||||
|
||||
until $(./mc admin decom status myminio/ | grep -q Complete); do
|
||||
echo "waiting for decom to finish..."
|
||||
sleep 1
|
||||
done
|
||||
|
||||
kill $pid_1
|
||||
kill $pid_2
|
||||
|
||||
sleep 5
|
||||
|
||||
(minio server --address ":9001" http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/removed.log) &
|
||||
pid=$!
|
||||
|
||||
export MC_HOST_myminio="http://minioadmin:minioadmin@localhost:9001/"
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
decom_user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
decom_policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
if [ $user_count -ne $decom_user_count ]; then
|
||||
echo "BUG: original user count differs after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ $policy_count -ne $decom_policy_count ]; then
|
||||
echo "BUG: original policy count differs after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc version info myminio/versioned | grep -q "versioning is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected versioning enabled after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
got_checksum=$(./mc cat myminio/versioned/dsync/drwmutex.go | md5sum)
|
||||
if [ "${expected_checksum}" != "${got_checksum}" ]; then
|
||||
echo "BUG: decommission failed on encrypted objects: expected ${expected_checksum} got ${got_checksum}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc ls -r myminio/versioned >decommissioned_ns.txt
|
||||
./mc ls -r --versions myminio/versioned >decommissioned_ns_versions.txt
|
||||
|
||||
out=$(diff -qpruN expanded_ns.txt decommissioned_ns.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission: $out"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
out=$(diff -qpruN expanded_ns_versions.txt decommissioned_ns_versions.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission: $out"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://127.0.0.1:9001/ -bucket versioned
|
||||
|
||||
kill $pid
|
||||
245
docs/distributed/decom-encrypted-kes.sh
Executable file
245
docs/distributed/decom-encrypted-kes.sh
Executable file
@@ -0,0 +1,245 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ -n "$TEST_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
|
||||
pkill minio
|
||||
pkill kes
|
||||
rm -rf /tmp/xl
|
||||
|
||||
if [ ! -f ./mc ]; then
|
||||
wget --quiet -O mc https://dl.minio.io/client/mc/release/linux-amd64/mc &&
|
||||
chmod +x mc
|
||||
fi
|
||||
|
||||
if [ ! -f ./kes ]; then
|
||||
wget --quiet -O kes https://github.com/minio/kes/releases/latest/download/kes-linux-amd64 &&
|
||||
chmod +x kes
|
||||
fi
|
||||
|
||||
if ! openssl version &>/dev/null; then
|
||||
apt install openssl || sudo apt install opensssl
|
||||
fi
|
||||
|
||||
# Start KES Server
|
||||
(./kes server --dev 2>&1 >kes-server.log) &
|
||||
kes_pid=$!
|
||||
sleep 5s
|
||||
API_KEY=$(grep "API Key" <kes-server.log | awk -F" " '{print $3}')
|
||||
(openssl s_client -connect 127.0.0.1:7373 2>/dev/null 1>public.crt)
|
||||
|
||||
export CI=true
|
||||
export MINIO_KMS_KES_ENDPOINT=https://127.0.0.1:7373
|
||||
export MINIO_KMS_KES_API_KEY="${API_KEY}"
|
||||
export MINIO_KMS_KES_KEY_NAME=minio-default-key
|
||||
export MINIO_KMS_KES_CAPATH=public.crt
|
||||
export MC_HOST_myminio="http://minioadmin:minioadmin@localhost:9000/"
|
||||
|
||||
(minio server http://localhost:9000/tmp/xl/{1...10}/disk{0...1} 2>&1 >/dev/null) &
|
||||
pid=$!
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
./mc admin user add myminio/ minio123 minio123
|
||||
./mc admin user add myminio/ minio12345 minio12345
|
||||
|
||||
./mc admin policy create myminio/ rw ./docs/distributed/rw.json
|
||||
./mc admin policy create myminio/ lake ./docs/distributed/rw.json
|
||||
|
||||
./mc admin policy attach myminio/ rw --user=minio123
|
||||
./mc admin policy attach myminio/ lake --user=minio12345
|
||||
|
||||
./mc mb -l myminio/versioned
|
||||
./mc mb -l myminio/versioned-1
|
||||
|
||||
./mc encrypt set sse-s3 myminio/versioned
|
||||
./mc encrypt set sse-kms minio-default-key myminio/versioned-1
|
||||
|
||||
./mc mirror internal myminio/versioned/ --quiet >/dev/null
|
||||
./mc mirror internal myminio/versioned-1/ --quiet >/dev/null
|
||||
|
||||
## Soft delete (creates delete markers)
|
||||
./mc rm -r --force myminio/versioned >/dev/null
|
||||
./mc rm -r --force myminio/versioned-1 >/dev/null
|
||||
|
||||
## mirror again to create another set of version on top
|
||||
./mc mirror internal myminio/versioned/ --quiet >/dev/null
|
||||
./mc mirror internal myminio/versioned-1/ --quiet >/dev/null
|
||||
|
||||
expected_checksum=$(./mc cat internal/dsync/drwmutex.go | md5sum)
|
||||
|
||||
user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
kill $pid
|
||||
|
||||
(minio server http://localhost:9000/tmp/xl/{1...10}/disk{0...1} http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/expanded_1.log) &
|
||||
pid_1=$!
|
||||
|
||||
(minio server --address ":9001" http://localhost:9000/tmp/xl/{1...10}/disk{0...1} http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/expanded_2.log) &
|
||||
pid_2=$!
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
expanded_user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
expanded_policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
if [ "$user_count" -ne "$expanded_user_count" ]; then
|
||||
echo "BUG: original user count differs from expanded setup"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$policy_count" -ne "$expanded_policy_count" ]; then
|
||||
echo "BUG: original policy count differs from expanded setup"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc version info myminio/versioned | grep -q "versioning is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "expected versioning enabled after expansion"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc encrypt info myminio/versioned | grep -q "Auto encryption 'sse-s3' is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "expected encryption enabled after expansion"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc version info myminio/versioned-1 | grep -q "versioning is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "expected versioning enabled after expansion"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc encrypt info myminio/versioned-1 | grep -q "Auto encryption 'sse-kms' is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "expected encryption enabled after expansion"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc mirror cmd myminio/versioned/ --quiet >/dev/null
|
||||
./mc mirror cmd myminio/versioned-1/ --quiet >/dev/null
|
||||
|
||||
./mc ls -r myminio/versioned/ >expanded_ns.txt
|
||||
./mc ls -r --versions myminio/versioned/ >expanded_ns_versions.txt
|
||||
./mc ls -r myminio/versioned-1/ >expanded_ns_1.txt
|
||||
./mc ls -r --versions myminio/versioned-1/ >expanded_ns_versions_1.txt
|
||||
|
||||
./mc admin decom start myminio/ http://localhost:9000/tmp/xl/{1...10}/disk{0...1}
|
||||
|
||||
until $(./mc admin decom status myminio/ | grep -q Complete); do
|
||||
echo "waiting for decom to finish..."
|
||||
sleep 1s
|
||||
done
|
||||
|
||||
kill $pid_1
|
||||
kill $pid_2
|
||||
|
||||
sleep 5s
|
||||
|
||||
(minio server --address ":9001" http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/removed.log) &
|
||||
pid=$!
|
||||
|
||||
sleep 30s
|
||||
|
||||
export MC_HOST_myminio="http://minioadmin:minioadmin@localhost:9001/"
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
decom_user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
decom_policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
if [ "$user_count" -ne "$decom_user_count" ]; then
|
||||
echo "BUG: original user count differs after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$policy_count" -ne "$decom_policy_count" ]; then
|
||||
echo "BUG: original policy count differs after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc version info myminio/versioned | grep -q "versioning is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected versioning enabled after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc encrypt info myminio/versioned | grep -q "Auto encryption 'sse-s3' is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected encryption enabled after expansion"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc version info myminio/versioned-1 | grep -q "versioning is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected versioning enabled after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc encrypt info myminio/versioned-1 | grep -q "Auto encryption 'sse-kms' is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected encryption enabled after expansion"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
got_checksum=$(./mc cat myminio/versioned/dsync/drwmutex.go | md5sum)
|
||||
if [ "${expected_checksum}" != "${got_checksum}" ]; then
|
||||
echo "BUG: decommission failed on encrypted objects: expected ${expected_checksum} got ${got_checksum}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
got_checksum_1=$(./mc cat myminio/versioned-1/dsync/drwmutex.go | md5sum)
|
||||
if [ "${expected_checksum}" != "${got_checksum_1}" ]; then
|
||||
echo "BUG: decommission failed on encrypted objects: expected ${expected_checksum} got ${got_checksum_1}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc ls -r myminio/versioned >decommissioned_ns.txt
|
||||
./mc ls -r --versions myminio/versioned >decommissioned_ns_versions.txt
|
||||
./mc ls -r myminio/versioned-1 >decommissioned_ns_1.txt
|
||||
./mc ls -r --versions myminio/versioned-1 >decommissioned_ns_versions_1.txt
|
||||
|
||||
out=$(diff -qpruN expanded_ns.txt decommissioned_ns.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission: $out"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
out=$(diff -qpruN expanded_ns_versions.txt decommissioned_ns_versions.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission: $out"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
out1=$(diff -qpruN expanded_ns_1.txt decommissioned_ns_1.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission: $out1"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
out1=$(diff -qpruN expanded_ns_versions_1.txt decommissioned_ns_versions_1.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission: $out1"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://127.0.0.1:9001/ -bucket versioned
|
||||
./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://127.0.0.1:9001/ -bucket versioned-1
|
||||
|
||||
kill $pid
|
||||
kill $kes_pid
|
||||
163
docs/distributed/decom-encrypted-sse-s3.sh
Executable file
163
docs/distributed/decom-encrypted-sse-s3.sh
Executable file
@@ -0,0 +1,163 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ -n "$TEST_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
|
||||
pkill minio
|
||||
rm -rf /tmp/xl
|
||||
|
||||
if [ ! -f ./mc ]; then
|
||||
wget --quiet -O mc https://dl.minio.io/client/mc/release/linux-amd64/mc &&
|
||||
chmod +x mc
|
||||
fi
|
||||
|
||||
export CI=true
|
||||
export MINIO_KMS_SECRET_KEY=my-minio-key:OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw=
|
||||
export MC_HOST_myminio="http://minioadmin:minioadmin@localhost:9000/"
|
||||
|
||||
(minio server http://localhost:9000/tmp/xl/{1...10}/disk{0...1} 2>&1 >/dev/null) &
|
||||
pid=$!
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
./mc admin user add myminio/ minio123 minio123
|
||||
./mc admin user add myminio/ minio12345 minio12345
|
||||
|
||||
./mc admin policy create myminio/ rw ./docs/distributed/rw.json
|
||||
./mc admin policy create myminio/ lake ./docs/distributed/rw.json
|
||||
|
||||
./mc admin policy attach myminio/ rw --user=minio123
|
||||
./mc admin policy attach myminio/ lake --user=minio12345
|
||||
|
||||
./mc mb -l myminio/versioned
|
||||
|
||||
./mc encrypt set sse-s3 myminio/versioned
|
||||
|
||||
./mc mirror internal myminio/versioned/ --quiet >/dev/null
|
||||
|
||||
## Soft delete (creates delete markers)
|
||||
./mc rm -r --force myminio/versioned >/dev/null
|
||||
|
||||
## mirror again to create another set of version on top
|
||||
./mc mirror internal myminio/versioned/ --quiet >/dev/null
|
||||
|
||||
expected_checksum=$(./mc cat internal/dsync/drwmutex.go | md5sum)
|
||||
|
||||
user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
kill $pid
|
||||
|
||||
(minio server http://localhost:9000/tmp/xl/{1...10}/disk{0...1} http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/expanded_1.log) &
|
||||
pid_1=$!
|
||||
|
||||
(minio server --address ":9001" http://localhost:9000/tmp/xl/{1...10}/disk{0...1} http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/expanded_2.log) &
|
||||
pid_2=$!
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
expanded_user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
expanded_policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
if [ $user_count -ne $expanded_user_count ]; then
|
||||
echo "BUG: original user count differs from expanded setup"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ $policy_count -ne $expanded_policy_count ]; then
|
||||
echo "BUG: original policy count differs from expanded setup"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc version info myminio/versioned | grep -q "versioning is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "expected versioning enabled after expansion"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc encrypt info myminio/versioned | grep -q "Auto encryption 'sse-s3' is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "expected encryption enabled after expansion"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc mirror cmd myminio/versioned/ --quiet >/dev/null
|
||||
|
||||
./mc ls -r myminio/versioned/ >expanded_ns.txt
|
||||
./mc ls -r --versions myminio/versioned/ >expanded_ns_versions.txt
|
||||
|
||||
./mc admin decom start myminio/ http://localhost:9000/tmp/xl/{1...10}/disk{0...1}
|
||||
|
||||
until $(./mc admin decom status myminio/ | grep -q Complete); do
|
||||
echo "waiting for decom to finish..."
|
||||
sleep 1
|
||||
done
|
||||
|
||||
kill $pid_1
|
||||
kill $pid_2
|
||||
|
||||
sleep 5
|
||||
|
||||
(minio server --address ":9001" http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/removed.log) &
|
||||
pid=$!
|
||||
|
||||
export MC_HOST_myminio="http://minioadmin:minioadmin@localhost:9001/"
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
decom_user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
decom_policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
if [ $user_count -ne $decom_user_count ]; then
|
||||
echo "BUG: original user count differs after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ $policy_count -ne $decom_policy_count ]; then
|
||||
echo "BUG: original policy count differs after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc version info myminio/versioned | grep -q "versioning is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected versioning enabled after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc encrypt info myminio/versioned | grep -q "Auto encryption 'sse-s3' is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected encryption enabled after expansion"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
got_checksum=$(./mc cat myminio/versioned/dsync/drwmutex.go | md5sum)
|
||||
if [ "${expected_checksum}" != "${got_checksum}" ]; then
|
||||
echo "BUG: decommission failed on encrypted objects: expected ${expected_checksum} got ${got_checksum}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc ls -r myminio/versioned >decommissioned_ns.txt
|
||||
./mc ls -r --versions myminio/versioned >decommissioned_ns_versions.txt
|
||||
|
||||
out=$(diff -qpruN expanded_ns.txt decommissioned_ns.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission: $out"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
out=$(diff -qpruN expanded_ns_versions.txt decommissioned_ns_versions.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission: $out"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://127.0.0.1:9001/ -bucket versioned
|
||||
|
||||
kill $pid
|
||||
149
docs/distributed/decom-encrypted.sh
Executable file
149
docs/distributed/decom-encrypted.sh
Executable file
@@ -0,0 +1,149 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ -n "$TEST_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
|
||||
pkill minio
|
||||
rm -rf /tmp/xl
|
||||
|
||||
if [ ! -f ./mc ]; then
|
||||
wget --quiet -O mc https://dl.minio.io/client/mc/release/linux-amd64/mc &&
|
||||
chmod +x mc
|
||||
fi
|
||||
|
||||
export CI=true
|
||||
export MINIO_KMS_AUTO_ENCRYPTION=on
|
||||
export MINIO_KMS_SECRET_KEY=my-minio-key:OSMM+vkKUTCvQs9YL/CVMIMt43HFhkUpqJxTmGl6rYw=
|
||||
|
||||
(minio server http://localhost:9000/tmp/xl/{1...10}/disk{0...1} 2>&1 >/dev/null) &
|
||||
pid=$!
|
||||
|
||||
export MC_HOST_myminio="http://minioadmin:minioadmin@localhost:9000/"
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
./mc admin user add myminio/ minio123 minio123
|
||||
./mc admin user add myminio/ minio12345 minio12345
|
||||
|
||||
./mc admin policy create myminio/ rw ./docs/distributed/rw.json
|
||||
./mc admin policy create myminio/ lake ./docs/distributed/rw.json
|
||||
|
||||
./mc admin policy attach myminio/ rw --user=minio123
|
||||
./mc admin policy attach myminio/ lake --user=minio12345
|
||||
|
||||
./mc mb -l myminio/versioned
|
||||
|
||||
./mc mirror internal myminio/versioned/ --quiet >/dev/null
|
||||
|
||||
## Soft delete (creates delete markers)
|
||||
./mc rm -r --force myminio/versioned >/dev/null
|
||||
|
||||
## mirror again to create another set of version on top
|
||||
./mc mirror internal myminio/versioned/ --quiet >/dev/null
|
||||
|
||||
expected_checksum=$(./mc cat internal/dsync/drwmutex.go | md5sum)
|
||||
|
||||
user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
kill $pid
|
||||
|
||||
(minio server http://localhost:9000/tmp/xl/{1...10}/disk{0...1} http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/expanded_1.log) &
|
||||
pid_1=$!
|
||||
|
||||
(minio server --address ":9001" http://localhost:9000/tmp/xl/{1...10}/disk{0...1} http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/expanded_2.log) &
|
||||
pid_2=$!
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
expanded_user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
expanded_policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
if [ $user_count -ne $expanded_user_count ]; then
|
||||
echo "BUG: original user count differs from expanded setup"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ $policy_count -ne $expanded_policy_count ]; then
|
||||
echo "BUG: original policy count differs from expanded setup"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc version info myminio/versioned | grep -q "versioning is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "expected versioning enabled after expansion"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc mirror cmd myminio/versioned/ --quiet >/dev/null
|
||||
|
||||
./mc ls -r myminio/versioned/ >expanded_ns.txt
|
||||
./mc ls -r --versions myminio/versioned/ >expanded_ns_versions.txt
|
||||
|
||||
./mc admin decom start myminio/ http://localhost:9000/tmp/xl/{1...10}/disk{0...1}
|
||||
|
||||
until $(./mc admin decom status myminio/ | grep -q Complete); do
|
||||
echo "waiting for decom to finish..."
|
||||
sleep 1
|
||||
done
|
||||
|
||||
kill $pid_1
|
||||
kill $pid_2
|
||||
|
||||
sleep 5
|
||||
|
||||
(minio server --address ":9001" http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/removed.log) &
|
||||
pid=$!
|
||||
|
||||
export MC_HOST_myminio="http://minioadmin:minioadmin@localhost:9001/"
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
decom_user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
decom_policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
if [ $user_count -ne $decom_user_count ]; then
|
||||
echo "BUG: original user count differs after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ $policy_count -ne $decom_policy_count ]; then
|
||||
echo "BUG: original policy count differs after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc version info myminio/versioned | grep -q "versioning is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected versioning enabled after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc ls -r myminio/versioned >decommissioned_ns.txt
|
||||
./mc ls -r --versions myminio/versioned >decommissioned_ns_versions.txt
|
||||
|
||||
out=$(diff -qpruN expanded_ns.txt decommissioned_ns.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission: $out"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
out=$(diff -qpruN expanded_ns_versions.txt decommissioned_ns_versions.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission: $out"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
got_checksum=$(./mc cat myminio/versioned/dsync/drwmutex.go | md5sum)
|
||||
if [ "${expected_checksum}" != "${got_checksum}" ]; then
|
||||
echo "BUG: decommission failed on encrypted objects: expected ${expected_checksum} got ${got_checksum}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://127.0.0.1:9001/ -bucket versioned
|
||||
|
||||
kill $pid
|
||||
218
docs/distributed/decom.sh
Executable file
218
docs/distributed/decom.sh
Executable file
@@ -0,0 +1,218 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ -n "$TEST_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
|
||||
pkill minio
|
||||
rm -rf /tmp/xl
|
||||
rm -rf /tmp/xltier
|
||||
|
||||
if [ ! -f ./mc ]; then
|
||||
wget --quiet -O mc https://dl.minio.io/client/mc/release/linux-amd64/mc &&
|
||||
chmod +x mc
|
||||
fi
|
||||
|
||||
export CI=true
|
||||
export MINIO_SCANNER_SPEED=fastest
|
||||
|
||||
(minio server http://localhost:9000/tmp/xl/{1...10}/disk{0...1} 2>&1 >/tmp/decom.log) &
|
||||
pid=$!
|
||||
|
||||
export MC_HOST_myminio="http://minioadmin:minioadmin@localhost:9000/"
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
./mc admin user add myminio/ minio123 minio123
|
||||
./mc admin user add myminio/ minio12345 minio12345
|
||||
|
||||
./mc admin policy create myminio/ rw ./docs/distributed/rw.json
|
||||
./mc admin policy create myminio/ lake ./docs/distributed/rw.json
|
||||
|
||||
./mc admin policy attach myminio/ rw --user=minio123
|
||||
./mc admin policy attach myminio/ lake --user=minio12345
|
||||
|
||||
./mc mb -l myminio/versioned
|
||||
|
||||
./mc mirror internal myminio/versioned/ --quiet >/dev/null
|
||||
|
||||
## Soft delete (creates delete markers)
|
||||
./mc rm -r --force myminio/versioned >/dev/null
|
||||
|
||||
## mirror again to create another set of version on top
|
||||
./mc mirror internal myminio/versioned/ --quiet >/dev/null
|
||||
|
||||
expected_checksum=$(./mc cat internal/dsync/drwmutex.go | md5sum)
|
||||
|
||||
user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
## create a warm tier instance
|
||||
(minio server /tmp/xltier/{1...4}/disk{0...1} --address :9002 2>&1 >/dev/null) &
|
||||
|
||||
export MC_HOST_mytier="http://minioadmin:minioadmin@localhost:9002/"
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
./mc mb -l myminio/bucket2
|
||||
./mc mb -l mytier/tiered
|
||||
|
||||
## create a tier and set up ilm policy to tier immediately
|
||||
./mc admin tier add minio myminio TIER1 --endpoint http://localhost:9002 --access-key minioadmin --secret-key minioadmin --bucket tiered --prefix prefix5/
|
||||
./mc ilm add myminio/bucket2 --transition-days 0 --transition-tier TIER1 --transition-days 0
|
||||
|
||||
## mirror some content to bucket2 and capture versions tiered
|
||||
./mc mirror internal myminio/bucket2/ --quiet >/dev/null
|
||||
./mc ls -r myminio/bucket2/ >bucket2_ns.txt
|
||||
./mc ls -r --versions myminio/bucket2/ >bucket2_ns_versions.txt
|
||||
|
||||
sleep 30
|
||||
|
||||
./mc ls -r --versions mytier/tiered/ >tiered_ns_versions.txt
|
||||
|
||||
kill $pid
|
||||
|
||||
(minio server http://localhost:9000/tmp/xl/{1...10}/disk{0...1} http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/expanded_1.log) &
|
||||
pid_1=$!
|
||||
|
||||
(minio server --address ":9001" http://localhost:9000/tmp/xl/{1...10}/disk{0...1} http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/expanded_2.log) &
|
||||
pid_2=$!
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
expanded_user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
expanded_policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
if [ $user_count -ne $expanded_user_count ]; then
|
||||
echo "BUG: original user count differs from expanded setup"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ $policy_count -ne $expanded_policy_count ]; then
|
||||
echo "BUG: original policy count differs from expanded setup"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc version info myminio/versioned | grep -q "versioning is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "expected versioning enabled after expansion"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc mirror cmd myminio/versioned/ --quiet >/dev/null
|
||||
|
||||
./mc ls -r myminio/versioned/ >expanded_ns.txt
|
||||
./mc ls -r --versions myminio/versioned/ >expanded_ns_versions.txt
|
||||
|
||||
./mc admin decom start myminio/ http://localhost:9000/tmp/xl/{1...10}/disk{0...1}
|
||||
|
||||
count=0
|
||||
until $(./mc admin decom status myminio/ | grep -q Complete); do
|
||||
echo "waiting for decom to finish..."
|
||||
count=$((count + 1))
|
||||
if [ ${count} -eq 120 ]; then
|
||||
./mc cat /tmp/expanded_*.log
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
|
||||
kill $pid_1
|
||||
kill $pid_2
|
||||
|
||||
sleep 5
|
||||
|
||||
(minio server --address ":9001" http://localhost:9001/tmp/xl/{11...30}/disk{0...3} 2>&1 >/tmp/removed.log) &
|
||||
pid=$!
|
||||
|
||||
sleep 5
|
||||
export MC_HOST_myminio="http://minioadmin:minioadmin@localhost:9001/"
|
||||
|
||||
./mc ready myminio
|
||||
|
||||
decom_user_count=$(./mc admin user list myminio/ | wc -l)
|
||||
decom_policy_count=$(./mc admin policy list myminio/ | wc -l)
|
||||
|
||||
if [ $user_count -ne $decom_user_count ]; then
|
||||
echo "BUG: original user count differs after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ $policy_count -ne $decom_policy_count ]; then
|
||||
echo "BUG: original policy count differs after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc version info myminio/versioned | grep -q "versioning is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected versioning enabled after decommission"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc ls -r myminio/versioned >decommissioned_ns.txt
|
||||
./mc ls -r --versions myminio/versioned >decommissioned_ns_versions.txt
|
||||
|
||||
out=$(diff -qpruN expanded_ns.txt decommissioned_ns.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission: $out"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
out=$(diff -qpruN expanded_ns_versions.txt decommissioned_ns_versions.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission: $out"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
got_checksum=$(./mc cat myminio/versioned/dsync/drwmutex.go | md5sum)
|
||||
if [ "${expected_checksum}" != "${got_checksum}" ]; then
|
||||
echo "BUG: decommission failed on encrypted objects: expected ${expected_checksum} got ${got_checksum}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# after decommissioning, compare listings in bucket2 and tiered
|
||||
./mc version info myminio/bucket2 | grep -q "versioning is enabled"
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected versioning enabled after decommission on bucket2"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc ls -r myminio/bucket2 >decommissioned_bucket2_ns.txt
|
||||
./mc ls -r --versions myminio/bucket2 >decommissioned_bucket2_ns_versions.txt
|
||||
./mc ls -r --versions mytier/tiered/ >tiered_ns_versions2.txt
|
||||
|
||||
out=$(diff -qpruN bucket2_ns.txt decommissioned_bucket2_ns.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission in bucket2: $out"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
out=$(diff -qpruN bucket2_ns_versions.txt decommissioned_bucket2_ns_versions.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission in bucket2x: $out"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
out=$(diff -qpruN tiered_ns_versions.txt tiered_ns_versions2.txt)
|
||||
ret=$?
|
||||
if [ $ret -ne 0 ]; then
|
||||
echo "BUG: expected no missing entries after decommission in warm tier: $out"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
got_checksum=$(./mc cat myminio/bucket2/dsync/drwmutex.go | md5sum)
|
||||
if [ "${expected_checksum}" != "${got_checksum}" ]; then
|
||||
echo "BUG: decommission failed on encrypted objects with tiering: expected ${expected_checksum} got ${got_checksum}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://127.0.0.1:9001/ -bucket bucket2
|
||||
s3-check-md5 -versions -access-key minioadmin -secret-key minioadmin -endpoint http://127.0.0.1:9001/ -bucket versioned
|
||||
|
||||
kill $pid
|
||||
109
docs/distributed/distributed-from-config-file.sh
Executable file
109
docs/distributed/distributed-from-config-file.sh
Executable file
@@ -0,0 +1,109 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -e
|
||||
|
||||
cleanup() {
|
||||
echo "Cleaning up instances of MinIO"
|
||||
pkill minio || true
|
||||
pkill -9 minio || true
|
||||
rm -rf /tmp/xl/ || true
|
||||
rm -rf /tmp/minio.configfile.{1,2,3,4} || true
|
||||
}
|
||||
|
||||
cleanup
|
||||
|
||||
unset MINIO_KMS_KES_CERT_FILE
|
||||
unset MINIO_KMS_KES_KEY_FILE
|
||||
unset MINIO_KMS_KES_ENDPOINT
|
||||
unset MINIO_KMS_KES_KEY_NAME
|
||||
|
||||
export MINIO_CI_CD=1
|
||||
|
||||
if [ ! -f ./mc ]; then
|
||||
os="$(uname -s)"
|
||||
arch="$(uname -m)"
|
||||
case "${arch}" in
|
||||
"x86_64")
|
||||
arch="amd64"
|
||||
;;
|
||||
esac
|
||||
|
||||
wget -O mc https://dl.minio.io/client/mc/release/${os,,}-${arch,,}/mc &&
|
||||
chmod +x mc
|
||||
fi
|
||||
|
||||
for i in $(seq 1 4); do
|
||||
s3Port="$((9000 + i))"
|
||||
consolePort="$((s3Port + 1000))"
|
||||
|
||||
cat <<EOF >/tmp/minio.configfile.$i
|
||||
version: v1
|
||||
address: ':${s3Port}'
|
||||
console-address: ':${consolePort}'
|
||||
rootUser: 'minr0otUS2r'
|
||||
rootPassword: 'pBU94AGAY85e'
|
||||
pools: # Specify the nodes and drives with pools
|
||||
-
|
||||
- 'http://localhost:9001/tmp/xl/node9001/mnt/disk{1...4}/'
|
||||
- 'http://localhost:9002/tmp/xl/node9002/mnt/disk{1,2,3,4}/'
|
||||
-
|
||||
- 'http://localhost:9003/tmp/xl/node9003/mnt/disk{1...4}/'
|
||||
- 'http://localhost:9004/tmp/xl/node9004/mnt/disk1/'
|
||||
- 'http://localhost:9004/tmp/xl/node9004/mnt/disk2/'
|
||||
- 'http://localhost:9004/tmp/xl/node9004/mnt/disk3/'
|
||||
- 'http://localhost:9004/tmp/xl/node9004/mnt/disk4/'
|
||||
EOF
|
||||
done
|
||||
|
||||
minio server --config /tmp/minio.configfile.1 >/tmp/minio1_1.log 2>&1 &
|
||||
site1_pid=$!
|
||||
minio server --config /tmp/minio.configfile.2 >/tmp/minio2_1.log 2>&1 &
|
||||
site2_pid=$!
|
||||
minio server --config /tmp/minio.configfile.3 >/tmp/minio3_1.log 2>&1 &
|
||||
site3_pid=$!
|
||||
minio server --config /tmp/minio.configfile.4 >/tmp/minio4_1.log 2>&1 &
|
||||
site4_pid=$!
|
||||
|
||||
export MC_HOST_minio1=http://minr0otUS2r:pBU94AGAY85e@localhost:9001
|
||||
export MC_HOST_minio3=http://minr0otUS2r:pBU94AGAY85e@localhost:9003
|
||||
|
||||
./mc ready minio1
|
||||
./mc ready minio3
|
||||
|
||||
./mc mb minio1/testbucket
|
||||
# copy large upload to newbucket on minio1
|
||||
truncate -s 17M lrgfile
|
||||
expected_checksum=$(cat ./lrgfile | md5sum)
|
||||
|
||||
./mc cp ./lrgfile minio1/testbucket
|
||||
|
||||
actual_checksum=$(./mc cat minio3/testbucket/lrgfile | md5sum)
|
||||
|
||||
if [ "${expected_checksum}" != "${actual_checksum}" ]; then
|
||||
echo "unexpected object checksum, expected: ${expected_checksum} got: ${actual_checksum}"
|
||||
exit
|
||||
fi
|
||||
|
||||
# Compare the difference of the list of disks and their location, with the below expected output
|
||||
diff <(./mc admin info minio1 --json | jq -r '.info.servers[].drives[] | "\(.pool_index),\(.set_index),\(.disk_index) \(.endpoint)"' | sort) <(
|
||||
cat <<EOF
|
||||
0,0,0 http://localhost:9001/tmp/xl/node9001/mnt/disk1
|
||||
0,0,1 http://localhost:9002/tmp/xl/node9002/mnt/disk1
|
||||
0,0,2 http://localhost:9001/tmp/xl/node9001/mnt/disk2
|
||||
0,0,3 http://localhost:9002/tmp/xl/node9002/mnt/disk2
|
||||
0,0,4 http://localhost:9001/tmp/xl/node9001/mnt/disk3
|
||||
0,0,5 http://localhost:9002/tmp/xl/node9002/mnt/disk3
|
||||
0,0,6 http://localhost:9001/tmp/xl/node9001/mnt/disk4
|
||||
0,0,7 http://localhost:9002/tmp/xl/node9002/mnt/disk4
|
||||
1,0,0 http://localhost:9003/tmp/xl/node9003/mnt/disk1
|
||||
1,0,1 http://localhost:9004/tmp/xl/node9004/mnt/disk1
|
||||
1,0,2 http://localhost:9003/tmp/xl/node9003/mnt/disk2
|
||||
1,0,3 http://localhost:9004/tmp/xl/node9004/mnt/disk2
|
||||
1,0,4 http://localhost:9003/tmp/xl/node9003/mnt/disk3
|
||||
1,0,5 http://localhost:9004/tmp/xl/node9004/mnt/disk3
|
||||
1,0,6 http://localhost:9003/tmp/xl/node9003/mnt/disk4
|
||||
1,0,7 http://localhost:9004/tmp/xl/node9004/mnt/disk4
|
||||
EOF
|
||||
)
|
||||
|
||||
cleanup
|
||||
115
docs/distributed/iam-import-with-missing-entities.sh
Executable file
115
docs/distributed/iam-import-with-missing-entities.sh
Executable file
@@ -0,0 +1,115 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ -n "$TEST_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
|
||||
pkill minio
|
||||
docker rm -f $(docker ps -aq)
|
||||
rm -rf /tmp/ldap{1..4}
|
||||
rm -rf /tmp/ldap1{1..4}
|
||||
|
||||
if [ ! -f ./mc ]; then
|
||||
wget --quiet -O mc https://dl.minio.io/client/mc/release/linux-amd64/mc &&
|
||||
chmod +x mc
|
||||
fi
|
||||
|
||||
mc -v
|
||||
|
||||
# Start LDAP server
|
||||
echo "Copying docs/distributed/samples/bootstrap-complete.ldif => minio-iam-testing/ldap/50-bootstrap.ldif"
|
||||
cp docs/distributed/samples/bootstrap-complete.ldif minio-iam-testing/ldap/50-bootstrap.ldif || exit 1
|
||||
cd ./minio-iam-testing
|
||||
make docker-images
|
||||
make docker-run
|
||||
cd -
|
||||
|
||||
export MC_HOST_myminio="http://minioadmin:minioadmin@localhost:22000"
|
||||
export MC_HOST_myminio1="http://minioadmin:minioadmin@localhost:24000"
|
||||
|
||||
# Start MinIO instance
|
||||
export CI=true
|
||||
(minio server --address :22000 --console-address :10000 http://localhost:22000/tmp/ldap{1...4} 2>&1 >/dev/null) &
|
||||
sleep 30
|
||||
./mc ready myminio
|
||||
|
||||
./mc idp ldap add myminio server_addr=localhost:389 server_insecure=on \
|
||||
lookup_bind_dn=cn=admin,dc=min,dc=io lookup_bind_password=admin \
|
||||
user_dn_search_base_dn=dc=min,dc=io user_dn_search_filter="(uid=%s)" \
|
||||
group_search_base_dn=ou=swengg,dc=min,dc=io group_search_filter="(&(objectclass=groupOfNames)(member=%d))"
|
||||
|
||||
./mc admin service restart myminio --json
|
||||
./mc ready myminio
|
||||
./mc admin cluster iam import myminio docs/distributed/samples/myminio-iam-info.zip
|
||||
sleep 10
|
||||
|
||||
# Verify the list of users and service accounts from the import
|
||||
./mc admin user list myminio
|
||||
USER_COUNT=$(./mc admin user list myminio | wc -l)
|
||||
if [ "${USER_COUNT}" -ne 2 ]; then
|
||||
echo "BUG: Expected no of users: 2 Found: ${USER_COUNT}"
|
||||
exit 1
|
||||
fi
|
||||
./mc admin user svcacct list myminio "uid=bobfisher,ou=people,ou=hwengg,dc=min,dc=io" --json
|
||||
SVCACCT_COUNT_1=$(./mc admin user svcacct list myminio "uid=bobfisher,ou=people,ou=hwengg,dc=min,dc=io" --json | jq '.accessKey' | wc -l)
|
||||
if [ "${SVCACCT_COUNT_1}" -ne 2 ]; then
|
||||
echo "BUG: Expected svcacct count for 'uid=bobfisher,ou=people,ou=hwengg,dc=min,dc=io': 2. Found: ${SVCACCT_COUNT_1}"
|
||||
exit 1
|
||||
fi
|
||||
./mc admin user svcacct list myminio "uid=dillon,ou=people,ou=swengg,dc=min,dc=io" --json
|
||||
SVCACCT_COUNT_2=$(./mc admin user svcacct list myminio "uid=dillon,ou=people,ou=swengg,dc=min,dc=io" --json | jq '.accessKey' | wc -l)
|
||||
if [ "${SVCACCT_COUNT_2}" -ne 2 ]; then
|
||||
echo "BUG: Expected svcacct count for 'uid=dillon,ou=people,ou=swengg,dc=min,dc=io': 2. Found: ${SVCACCT_COUNT_2}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Kill MinIO and LDAP to start afresh with missing groups/DN
|
||||
pkill minio
|
||||
docker rm -f $(docker ps -aq)
|
||||
rm -rf /tmp/ldap{1..4}
|
||||
|
||||
# Deploy the LDAP config witg missing groups/DN
|
||||
echo "Copying docs/distributed/samples/bootstrap-partial.ldif => minio-iam-testing/ldap/50-bootstrap.ldif"
|
||||
cp docs/distributed/samples/bootstrap-partial.ldif minio-iam-testing/ldap/50-bootstrap.ldif || exit 1
|
||||
cd ./minio-iam-testing
|
||||
make docker-images
|
||||
make docker-run
|
||||
cd -
|
||||
|
||||
(minio server --address ":24000" --console-address :10000 http://localhost:24000/tmp/ldap1{1...4} 2>&1 >/dev/null) &
|
||||
sleep 30
|
||||
./mc ready myminio1
|
||||
|
||||
./mc idp ldap add myminio1 server_addr=localhost:389 server_insecure=on \
|
||||
lookup_bind_dn=cn=admin,dc=min,dc=io lookup_bind_password=admin \
|
||||
user_dn_search_base_dn=dc=min,dc=io user_dn_search_filter="(uid=%s)" \
|
||||
group_search_base_dn=ou=hwengg,dc=min,dc=io group_search_filter="(&(objectclass=groupOfNames)(member=%d))"
|
||||
|
||||
./mc admin service restart myminio1 --json
|
||||
./mc ready myminio1
|
||||
./mc admin cluster iam import myminio1 docs/distributed/samples/myminio-iam-info.zip
|
||||
sleep 10
|
||||
|
||||
# Verify the list of users and service accounts from the import
|
||||
./mc admin user list myminio1
|
||||
USER_COUNT=$(./mc admin user list myminio1 | wc -l)
|
||||
if [ "${USER_COUNT}" -ne 1 ]; then
|
||||
echo "BUG: Expected no of users: 1 Found: ${USER_COUNT}"
|
||||
exit 1
|
||||
fi
|
||||
./mc admin user svcacct list myminio1 "uid=bobfisher,ou=people,ou=hwengg,dc=min,dc=io" --json
|
||||
SVCACCT_COUNT_1=$(./mc admin user svcacct list myminio1 "uid=bobfisher,ou=people,ou=hwengg,dc=min,dc=io" --json | jq '.accessKey' | wc -l)
|
||||
if [ "${SVCACCT_COUNT_1}" -ne 2 ]; then
|
||||
echo "BUG: Expected svcacct count for 'uid=bobfisher,ou=people,ou=hwengg,dc=min,dc=io': 2. Found: ${SVCACCT_COUNT_1}"
|
||||
exit 1
|
||||
fi
|
||||
./mc admin user svcacct list myminio1 "uid=dillon,ou=people,ou=swengg,dc=min,dc=io" --json
|
||||
SVCACCT_COUNT_2=$(./mc admin user svcacct list myminio1 "uid=dillon,ou=people,ou=swengg,dc=min,dc=io" --json | jq '.accessKey' | wc -l)
|
||||
if [ "${SVCACCT_COUNT_2}" -ne 0 ]; then
|
||||
echo "BUG: Expected svcacct count for 'uid=dillon,ou=people,ou=swengg,dc=min,dc=io': 0. Found: ${SVCACCT_COUNT_2}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Finally kill running processes
|
||||
pkill minio
|
||||
docker rm -f $(docker ps -aq)
|
||||
82
docs/distributed/iam-import-with-openid.sh
Executable file
82
docs/distributed/iam-import-with-openid.sh
Executable file
@@ -0,0 +1,82 @@
|
||||
#!/bin/bash
|
||||
|
||||
if [ -n "$TEST_DEBUG" ]; then
|
||||
set -x
|
||||
fi
|
||||
|
||||
pkill minio
|
||||
docker rm -f $(docker ps -aq)
|
||||
rm -rf /tmp/openid{1..4}
|
||||
|
||||
export MC_HOST_myminio="http://minioadmin:minioadmin@localhost:22000"
|
||||
# The service account used below is already present in iam configuration getting imported
|
||||
export MC_HOST_myminio1="http://dillon-service-2:dillon-service-2@localhost:22000"
|
||||
|
||||
# Start MinIO instance
|
||||
export CI=true
|
||||
|
||||
if [ ! -f ./mc ]; then
|
||||
wget --quiet -O mc https://dl.minio.io/client/mc/release/linux-amd64/mc &&
|
||||
chmod +x mc
|
||||
fi
|
||||
|
||||
mc -v
|
||||
|
||||
# Start openid server
|
||||
(
|
||||
cd ./minio-iam-testing
|
||||
make docker-images
|
||||
make docker-run
|
||||
cd -
|
||||
)
|
||||
|
||||
(minio server --address :22000 --console-address :10000 http://localhost:22000/tmp/openid{1...4} 2>&1 >/tmp/server.log) &
|
||||
./mc ready myminio
|
||||
./mc mb myminio/test-bucket
|
||||
./mc cp /etc/hosts myminio/test-bucket
|
||||
|
||||
./mc idp openid add myminio \
|
||||
config_url="http://localhost:5556/dex/.well-known/openid-configuration" \
|
||||
client_id="minio-client-app" \
|
||||
client_secret="minio-client-app-secret" \
|
||||
scopes="openid,groups,email,profile" \
|
||||
redirect_uri="http://127.0.0.1:10000/oauth_callback" \
|
||||
display_name="Login via dex1" \
|
||||
role_policy="consoleAdmin"
|
||||
|
||||
./mc admin service restart myminio --json
|
||||
./mc ready myminio
|
||||
./mc admin cluster iam import myminio docs/distributed/samples/myminio-iam-info-openid.zip
|
||||
|
||||
# Verify if buckets / objects accessible using service account
|
||||
echo "Verifying buckets and objects access for the imported service account"
|
||||
|
||||
./mc ls myminio1/ --json
|
||||
BKT_COUNT=$(./mc ls myminio1/ --json | jq '.key' | wc -l)
|
||||
if [ "${BKT_COUNT}" -ne 1 ]; then
|
||||
echo "BUG: Expected no of bucket: 1, Found: ${BKT_COUNT}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
BKT_NAME=$(./mc ls myminio1/ --json | jq '.key' | sed 's/"//g' | sed 's\/\\g')
|
||||
if [[ ${BKT_NAME} != "test-bucket" ]]; then
|
||||
echo "BUG: Expected bucket: test-bucket, Found: ${BKT_NAME}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
./mc ls myminio1/test-bucket
|
||||
OBJ_COUNT=$(./mc ls myminio1/test-bucket --json | jq '.key' | wc -l)
|
||||
if [ "${OBJ_COUNT}" -ne 1 ]; then
|
||||
echo "BUG: Expected no of objects: 1, Found: ${OBJ_COUNT}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
OBJ_NAME=$(./mc ls myminio1/test-bucket --json | jq '.key' | sed 's/"//g')
|
||||
if [[ ${OBJ_NAME} != "hosts" ]]; then
|
||||
echo "BUG: Expected object: hosts, Found: ${BKT_NAME}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Finally kill running processes
|
||||
pkill minio
|
||||
docker rm -f $(docker ps -aq)
|
||||
14
docs/distributed/rw.json
Normal file
14
docs/distributed/rw.json
Normal file
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"s3:*"
|
||||
],
|
||||
"Resource": [
|
||||
"arn:aws:s3:::*"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
123
docs/distributed/samples/bootstrap-complete.ldif
Normal file
123
docs/distributed/samples/bootstrap-complete.ldif
Normal file
@@ -0,0 +1,123 @@
|
||||
# Create hardware engg org unit
|
||||
dn: ou=hwengg,dc=min,dc=io
|
||||
objectClass: organizationalUnit
|
||||
ou: hwengg
|
||||
|
||||
# Create people sub-org
|
||||
dn: ou=people,ou=hwengg,dc=min,dc=io
|
||||
objectClass: organizationalUnit
|
||||
ou: people
|
||||
|
||||
# Create Alice, Bob and Cody in hwengg
|
||||
dn: uid=alice1,ou=people,ou=hwengg,dc=min,dc=io
|
||||
objectClass: inetOrgPerson
|
||||
cn: Alice Smith
|
||||
sn: Smith
|
||||
uid: alice1
|
||||
mail: alice@example.io
|
||||
userPassword: {SSHA}Yeh2/IV/q/HjG2yzN3YdE9CAF3EJFCLu
|
||||
|
||||
dn: uid=bobfisher,ou=people,ou=hwengg,dc=min,dc=io
|
||||
objectClass: inetOrgPerson
|
||||
cn: Robert Fisher
|
||||
sn: Fisher
|
||||
uid: bobfisher
|
||||
mail: bob@example.io
|
||||
userPassword: {SSHA}LktfbhK5oXSdDWCNzauJ9JA+Poxinl3y
|
||||
|
||||
dn: uid=cody3,ou=people,ou=hwengg,dc=min,dc=io
|
||||
objectClass: inetOrgPerson
|
||||
cn: Cody Thomas
|
||||
sn: Thomas
|
||||
uid: cody3
|
||||
mail: cody@example.io
|
||||
userPassword: {SSHA}H8B0gaOd4bRklK3fXj9ltHvJXWQFXW5Q
|
||||
|
||||
# Create groups ou for hwengg
|
||||
dn: ou=groups,ou=hwengg,dc=min,dc=io
|
||||
objectclass: organizationalUnit
|
||||
ou: groups
|
||||
description: groups branch
|
||||
|
||||
# Create project groups
|
||||
|
||||
dn: cn=projectx,ou=groups,ou=hwengg,dc=min,dc=io
|
||||
objectclass: groupofnames
|
||||
cn: projectx
|
||||
description: Project X group members
|
||||
member: uid=alice1,ou=people,ou=hwengg,dc=min,dc=io
|
||||
member: uid=bobfisher,ou=people,ou=hwengg,dc=min,dc=io
|
||||
|
||||
dn: cn=projecty,ou=groups,ou=hwengg,dc=min,dc=io
|
||||
objectclass: groupofnames
|
||||
cn: projecty
|
||||
description: Project Y group members
|
||||
member: uid=bobfisher,ou=people,ou=hwengg,dc=min,dc=io
|
||||
member: uid=cody3,ou=people,ou=hwengg,dc=min,dc=io
|
||||
|
||||
# Create software engg org unit
|
||||
dn: ou=swengg,dc=min,dc=io
|
||||
objectClass: organizationalUnit
|
||||
ou: swengg
|
||||
|
||||
# Create people sub-org
|
||||
dn: ou=people,ou=swengg,dc=min,dc=io
|
||||
objectClass: organizationalUnit
|
||||
ou: people
|
||||
|
||||
# Create Dillon, Elizabeth and Fahim in swengg
|
||||
dn: uid=dillon,ou=people,ou=swengg,dc=min,dc=io
|
||||
objectClass: inetOrgPerson
|
||||
cn: Dillon Harper
|
||||
sn: Harper
|
||||
uid: dillon
|
||||
mail: dillon@example.io
|
||||
userPassword: {SSHA}UH+LmoEhWWW6s9rjgdpqHPI0qCMouY8+
|
||||
|
||||
dn: uid=liza,ou=people,ou=swengg,dc=min,dc=io
|
||||
objectClass: inetOrgPerson
|
||||
cn: Elizabeth Jones
|
||||
sn: Jones
|
||||
uid: liza
|
||||
mail: ejones@example.io
|
||||
userPassword: {SSHA}feVkKkafHtsu2Io7n0tQP4Cnh8/Oy1PK
|
||||
|
||||
dn: uid=fahim,ou=people,ou=swengg,dc=min,dc=io
|
||||
objectClass: inetOrgPerson
|
||||
cn: Fahim Ahmed
|
||||
sn: Ahmed
|
||||
uid: fahim
|
||||
mail: fahmed@example.io
|
||||
userPassword: {SSHA}lRNH+PHooRaruiEb+CBEA21EZLMkAmcc
|
||||
|
||||
# Add a user with special chars. The password = example here.
|
||||
dn: uid=Пользователь,OU=people,OU=swengg,DC=min,DC=io
|
||||
objectClass: inetOrgPerson
|
||||
cn: Special Charsman
|
||||
sn: Charsman
|
||||
uid: Пользователь
|
||||
mail: scharsman@example.io
|
||||
userPassword: {SSHA}XQSZqLPvYgm30wR7pk67a1GW+q+DDvSj
|
||||
|
||||
# Creates groups ou for swengg
|
||||
dn: ou=groups,ou=swengg,dc=min,dc=io
|
||||
objectclass: organizationalUnit
|
||||
ou: groups
|
||||
description: groups branch
|
||||
|
||||
# Create project groups
|
||||
|
||||
dn: cn=projecta,ou=groups,ou=swengg,dc=min,dc=io
|
||||
objectclass: groupofnames
|
||||
cn: projecta
|
||||
description: Project A group members
|
||||
member: uid=dillon,ou=people,ou=swengg,dc=min,dc=io
|
||||
|
||||
dn: cn=projectb,ou=groups,ou=swengg,dc=min,dc=io
|
||||
objectclass: groupofnames
|
||||
cn: projectb
|
||||
description: Project B group members
|
||||
member: uid=dillon,ou=people,ou=swengg,dc=min,dc=io
|
||||
member: uid=liza,ou=people,ou=swengg,dc=min,dc=io
|
||||
member: uid=fahim,ou=people,ou=swengg,dc=min,dc=io
|
||||
member: uid=Пользователь,OU=people,OU=swengg,DC=min,DC=io
|
||||
56
docs/distributed/samples/bootstrap-partial.ldif
Normal file
56
docs/distributed/samples/bootstrap-partial.ldif
Normal file
@@ -0,0 +1,56 @@
|
||||
# Create hardware engg org unit
|
||||
dn: ou=hwengg,dc=min,dc=io
|
||||
objectClass: organizationalUnit
|
||||
ou: hwengg
|
||||
|
||||
# Create people sub-org
|
||||
dn: ou=people,ou=hwengg,dc=min,dc=io
|
||||
objectClass: organizationalUnit
|
||||
ou: people
|
||||
|
||||
# Create Alice, Bob and Cody in hwengg
|
||||
dn: uid=alice1,ou=people,ou=hwengg,dc=min,dc=io
|
||||
objectClass: inetOrgPerson
|
||||
cn: Alice Smith
|
||||
sn: Smith
|
||||
uid: alice1
|
||||
mail: alice@example.io
|
||||
userPassword: {SSHA}Yeh2/IV/q/HjG2yzN3YdE9CAF3EJFCLu
|
||||
|
||||
dn: uid=bobfisher,ou=people,ou=hwengg,dc=min,dc=io
|
||||
objectClass: inetOrgPerson
|
||||
cn: Robert Fisher
|
||||
sn: Fisher
|
||||
uid: bobfisher
|
||||
mail: bob@example.io
|
||||
userPassword: {SSHA}LktfbhK5oXSdDWCNzauJ9JA+Poxinl3y
|
||||
|
||||
dn: uid=cody3,ou=people,ou=hwengg,dc=min,dc=io
|
||||
objectClass: inetOrgPerson
|
||||
cn: Cody Thomas
|
||||
sn: Thomas
|
||||
uid: cody3
|
||||
mail: cody@example.io
|
||||
userPassword: {SSHA}H8B0gaOd4bRklK3fXj9ltHvJXWQFXW5Q
|
||||
|
||||
# Create groups ou for hwengg
|
||||
dn: ou=groups,ou=hwengg,dc=min,dc=io
|
||||
objectclass: organizationalUnit
|
||||
ou: groups
|
||||
description: groups branch
|
||||
|
||||
# Create project groups
|
||||
|
||||
dn: cn=projectx,ou=groups,ou=hwengg,dc=min,dc=io
|
||||
objectclass: groupofnames
|
||||
cn: projectx
|
||||
description: Project X group members
|
||||
member: uid=alice1,ou=people,ou=hwengg,dc=min,dc=io
|
||||
member: uid=bobfisher,ou=people,ou=hwengg,dc=min,dc=io
|
||||
|
||||
dn: cn=projecty,ou=groups,ou=hwengg,dc=min,dc=io
|
||||
objectclass: groupofnames
|
||||
cn: projecty
|
||||
description: Project Y group members
|
||||
member: uid=bobfisher,ou=people,ou=hwengg,dc=min,dc=io
|
||||
member: uid=cody3,ou=people,ou=hwengg,dc=min,dc=io
|
||||
BIN
docs/distributed/samples/myminio-iam-info-openid.zip
Normal file
BIN
docs/distributed/samples/myminio-iam-info-openid.zip
Normal file
Binary file not shown.
BIN
docs/distributed/samples/myminio-iam-info.zip
Normal file
BIN
docs/distributed/samples/myminio-iam-info.zip
Normal file
Binary file not shown.
Reference in New Issue
Block a user