Setup Scylla Cluster on CentOS 7

Scylla Is Next Generation NoSQL. Power your mission-critical applications with the best traits of Apache Cassandra at 10x the performance and low tail-latency at all times.

Prequisite

  • CentOS 7.2 or later, for the 64-bit x86_64 architecture.
  • ABRT conflict with Scylla coredump configuration. Remove it before installing Scylla: sudo yum remove -y abrt

Add Repo

Add the Scylla RPM repository to your system.

$ curl -o /etc/yum.repos.d/scylla.repo -L http://repositories.scylladb.com/scylla/repo/0de3e5d067167ebe832f17671ff55e78/centos/scylladb-2.0.repo

[root@db01 ~]# curl -o /etc/yum.repos.d/scylla.repo -L http://repositories.scylladb.com/scylla/repo/0de3e5d067167ebe832f17671ff55e78/centos/scylladb-2.0.repo
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
100 183 100 183 0 0 682 0 --:--:-- --:--:-- --:--:-- 682
100 933 100 933 0 0 518 0 0:00:01 0:00:01 --:--:-- 731
[root@db01 ~]#

Install

Use the below command to install Scylla.

$ yum install scylla

Confuguration

Configure the /etc/scylla/scylla.yaml file with the following parameters.

Item                 Content
=====                ========
cluster_name         Name of the cluster, all the nodes in the cluster must have the same name
seeds                Seed nodes are used during startup to bootstrap the gossip process and join the cluster
listen_address       IP address that the Scylla use to connect to other Scylla nodes in the cluster
rpc_address          IP address of interface for client connections (Thrift, CQL)

cluster_name: 'Test Cluster'
seed_provider:
       - class_name: org.apache.cassandra.locator.SimpleSeedProvider
         parameters:
               - seeds: "127.0.0.1"
listen_address: localhost
rpc_address: localhost

Run the scylla_setup script to tune the system settings.

$ sudo scylla_setup

[root@db01 ~]# scylla_setup
Skip any of the following steps by answering 'no'
Do you want to run kernel version check?
Answer yes to have this script verify that the currently installed kernel is qualified to run Scylla; answer no to skip this check.
[YES/no]y
This is a supported kernel version.
Do you want to verify ScyllaDB packages installed?
Answer yes to have this script check that ScyllaDB is already installed; answer no to skip this check.
[YES/no]y
Do you want to enable ScyllaDB services?
Answer yes to automatically start Scylla when the node boots; answer no to skip this step.
[YES/no]no
Do you want to disable SELinux?
Answer yes to disable SELinux and improve performance; answer no to keep it activated.
[YES/no]y
Do you want to setup NTP?
Answer yes to enable time synchronization at boot time. This keeps time right on the node. Answer no to do nothing.
[YES/no]y
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.excellmedia.net
 * epel: epel.mirror.angkasa.id
 * extras: centos.excellmedia.net
 * updates: centos.excellmedia.net
Resolving Dependencies
--> Running transaction check
---> Package ntp.x86_64 0:4.2.6p5-25.el7.centos.2 will be installed
--> Processing Dependency: libopts.so.25()(64bit) for package: ntp-4.2.6p5-25.el7.centos.2.x86_64
---> Package ntpdate.x86_64 0:4.2.6p5-25.el7.centos.2 will be installed
--> Running transaction check
---> Package autogen-libopts.x86_64 0:5.18-5.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================
 Package                                 Arch                           Version                                            Repository                    Size
==============================================================================================================================================================
Installing:
 ntp                                     x86_64                         4.2.6p5-25.el7.centos.2                            base                         547 k
 ntpdate                                 x86_64                         4.2.6p5-25.el7.centos.2                            base                          86 k
Installing for dependencies:
 autogen-libopts                         x86_64                         5.18-5.el7                                         base                          66 k

Transaction Summary
==============================================================================================================================================================
Install  2 Packages (+1 Dependent package)

Total download size: 699 k
Installed size: 1.6 M
Downloading packages:
(1/3): ntpdate-4.2.6p5-25.el7.centos.2.x86_64.rpm                                                                                      |  86 kB  00:00:01
(2/3): autogen-libopts-5.18-5.el7.x86_64.rpm                                                                                           |  66 kB  00:00:01
(3/3): ntp-4.2.6p5-25.el7.centos.2.x86_64.rpm                                                                                          | 547 kB  00:00:04
--------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                         151 kB/s | 699 kB  00:00:04
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : autogen-libopts-5.18-5.el7.x86_64                                                                                                          1/3
  Installing : ntpdate-4.2.6p5-25.el7.centos.2.x86_64                                                                                                     2/3
  Installing : ntp-4.2.6p5-25.el7.centos.2.x86_64                                                                                                         3/3
  Verifying  : ntp-4.2.6p5-25.el7.centos.2.x86_64                                                                                                         1/3
  Verifying  : ntpdate-4.2.6p5-25.el7.centos.2.x86_64                                                                                                     2/3
  Verifying  : autogen-libopts-5.18-5.el7.x86_64                                                                                                          3/3

Installed:
  ntp.x86_64 0:4.2.6p5-25.el7.centos.2                                        ntpdate.x86_64 0:4.2.6p5-25.el7.centos.2

Dependency Installed:
  autogen-libopts.x86_64 0:5.18-5.el7

Complete!
 2 Feb 11:51:55 ntpdate[11696]: step time server 52.66.5.185 offset 37044.150056 sec
Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
Do you want to setup RAID and XFS?
It is recommended to use RAID0 and XFS for Scylla data. If you select yes, you will be prompt to choose which unmounted disks to use for Scylla data. Selected disks will be formatted in the process.
[YES/no]no
Do you want to setup coredump?
Answer yes to enable core dumps; this allows to do post-mortem analysis of Scylla state after a crash. Answer no to do nothing.
[YES/no]y
kernel.core_pattern = |/usr/lib/systemd/systemd-coredump %p %u %g %s %t %e
Do you want to setup sysconfig?
Answer yes to do system wide configuration customized for Scylla. Answer no to do nothing.
[YES/no]y
Please select NIC from following list:
enp0s3 enp0s8
> enps08
enp0s3 enp0s8
> enp0s8
Do you want to optimize NIC queue settings?
Answer yes to enable network card optimization and improve performance. Answer no to skip this optimization.
[YES/no]yes
Setting parameters on /etc/sysconfig/scylla-server
Do you want to setup IO configuration?
Answer yes to let iotune study what are your disks IO profile and adapt Scylla to it. Answer no to skip this action.
[YES/no]y
Generating evaluation file sized 10GB...10GB written in 19 seconds
Refining search for maximum. So far, 36996 IOPS
Maximum throughput: 36996 IOPS
Recommended --max-io-requests: 27
Written the above values to /etc/scylla.d/io.conf
Do you want to install node exporter and export Prometheus data from the node?
Answer yes to install it; answer no to skip this installation.
[YES/no]y
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   628    0   628    0     0    467      0 --:--:--  0:00:01 --:--:--   467
100 3925k  100 3925k    0     0   342k      0  0:00:11  0:00:11 --:--:--  792k
node_exporter-0.14.0.linux-amd64/
node_exporter-0.14.0.linux-amd64/LICENSE
node_exporter-0.14.0.linux-amd64/NOTICE
node_exporter-0.14.0.linux-amd64/node_exporter
Created symlink from /etc/systemd/system/multi-user.target.wants/node-exporter.service to /usr/lib/systemd/system/node-exporter.service.
node_exporter successfully installed
Do you want to setup CPU scaling governor?
Answer yes to set CPU scaling governor to performance at boot time. Answer no to do nothing.
[YES/no]y
This computer doesn't supported CPU scaling configuration.
Do you want to enable fstrim service?
Answer yes to run fstrim on your SSD. Answer no to do nothing.
[YES/no]no
ScyllaDB setup finished.
Please restart machine before using ScyllaDB, since you have disabled
 SELinux.
[root@db01 ~]#

This script invokes a set of scripts to configure several operating system settings, like setting RAID0 and XFS filesystem. It also runs a short (up to a few minutes) benchmark on your storage and generates the /etc/scylla.d/io.conf configuration file. When the file is ready, you can start Scylla (see below). Scylla will not run without XFS or io.conf file. To bypass this check, set Scylla to developer mode

Run

Run Scylla as a service.

krishna@Ubuntu16:~$ sudo systemctl start scylla-server
Job for scylla-server.service failed because the control process exited with error code. See "systemctl status scylla-server.service" and "journalctl -xe" for details.

Scylla is showing some errors.

Sep 22 17:08:47 Ubuntu16 scylla_prepare[14795]: tuning: /sys/devices/pci0000:00/0000:00:0d.0/ata3/host2/target2:0:0/2:0:0:0/block/sda/queue/scheduler noop
Sep 22 17:08:47 Ubuntu16 scylla_prepare[14795]: tuning: /sys/devices/pci0000:00/0000:00:0d.0/ata3/host2/target2:0:0/2:0:0:0/block/sda/queue/nomerges 2
Sep 22 17:08:47 Ubuntu16 scylla_prepare[14795]: tuning /sys/devices/pci0000:00/0000:00:0d.0/ata3/host2/target2:0:0/2:0:0:0/block/sda/sda1
Sep 22 17:08:47 Ubuntu16 scylla[14800]: Scylla version 1.7.4-0.20170726.ff643e3 starting ...
Sep 22 17:08:47 Ubuntu16 scylla[14800]: [shard 0] init - Only 804 MiB per shard; this is below the recommended minimum of 1 GiB/shard; terminating.Configure more memory (--memory option)
Sep 22 17:08:47 Ubuntu16 scylla[14800]: [shard 0] seastar - Exiting on unhandled exception: std::runtime_error (configuration (memory per shard too low))
Sep 22 17:08:47 Ubuntu16 systemd[1]: scylla-server.service: Main process exited, code=exited, status=1/FAILURE
Sep 22 17:08:47 Ubuntu16 systemd[1]: Failed to start Scylla Server.
Sep 22 17:08:47 Ubuntu16 systemd[1]: scylla-server.service: Unit entered failed state.
Sep 22 17:08:47 Ubuntu16 systemd[1]: scylla-server.service: Failed with result 'exit-code'.

Change to developer mode by adding developer_mode: true in Scylla config file to fix the above issues. This should be done for testing purpose only.

developer_mode: true

Now, restart the Scylla service.

[root@db01 ~]# systemctl start scylla-server
[root@db01 ~]# systemctl status scylla-server
● scylla-server.service - Scylla Server
   Loaded: loaded (/usr/lib/systemd/system/scylla-server.service; disabled; vendor preset: disabled)
   Active: active (running) since Fri 2018-02-02 12:37:31 IST; 8s ago
  Process: 1075 ExecStartPre=/usr/lib/scylla/scylla_prepare (code=exited, status=0/SUCCESS)
 Main PID: 1086 (scylla)
   Status: "starting cf cache hit rate calculator"
   CGroup: /system.slice/scylla-server.service
           └─1086 /usr/bin/scylla --log-to-syslog 1 --log-to-stdout 0 --default-log-level info --collectd-address=127.0.0.1:25826 --collectd=1 --collectd-p...

Feb 02 12:37:32 db01 scylla[1086]:  [shard 0] gossip - Feature LARGE_PARTITIONS is enabled
Feb 02 12:37:32 db01 scylla[1086]:  [shard 0] gossip - Feature COUNTERS is enabled
Feb 02 12:37:32 db01 scylla[1086]:  [shard 0] gossip - Feature CORRECT_COUNTER_ORDER is enabled
Feb 02 12:37:32 db01 scylla[1086]:  [shard 0] gossip - Feature SCHEMA_TABLES_V3 is enabled
Feb 02 12:37:32 db01 scylla[1086]:  [shard 0] gossip - Feature CORRECT_NON_COMPOUND_RANGE_TOMBSTONES is enabled
Feb 02 12:37:32 db01 scylla[1086]:  [shard 0] storage_service - Using saved tokens {9195022591313047823, 9140147656116707974, 9021965674587313209...4703753735
Feb 02 12:37:32 db01 scylla[1086]:  [shard 0] compaction - Compacting [/data/scylla/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-6-Data.db...
Feb 02 12:37:32 db01 scylla[1086]:  [shard 0] compaction - Compacted 4 sstables to [/data/scylla/system/local-7ad54392bcdd35a684174e047860b377/sy...rged to 1.
Feb 02 12:37:32 db01 scylla[1086]:  [shard 0] storage_service - NORMAL: node is now in normal status
Feb 02 12:37:32 db01 scylla[1086]:  [shard 0] gossip - Waiting for gossip to settle before accepting client requests...
Hint: Some lines were ellipsized, use -l to show in full.
[root@db01 ~]#
[root@db01 ~]# nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address    Load       Tokens       Owns (effective)  Host ID                               Rack
UN  127.0.0.1  160.65 KB  256          100.0%            2e606483-d332-4e7b-ae66-8f68b5688a85  rack1

[root@db01 ~]#
[root@db01 ~]# cqlsh
Connected to Xfs_Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.0.8 | CQL spec 3.3.1 | Native protocol v4]
Use HELP for help.
cqlsh>
cqlsh>

Cluster Configuration

Add host entry to all the nodes of scylla cluster like below.

[root@db01 ~]# cat /etc/hosts
192.168.19.7    db01
192.168.19.10   db02
192.168.19.11   db03
[root@db01 ~]#

In the scylla.yaml file edit the parameters listed below. The file can be found under /etc/scylla/

  •  cluster_name – Set the selected cluster_name
  •  seeds – Set the selected seed nodes
  •  listen_address – IP address that Scylla used to connect to other Scylla nodes in the cluster
  •  auto_bootstrap – By default, this parameter is set to true, it allows new nodes to migrate data to themselves automatically. When creating new cluster, it is recommended to set this parameter to false
  •  endpoint_snitch – Set the selected snitch
  •  rpc_address – Address for client connection (Thrift, CQL)

Installing Three Scylla nodes the IP’s are

192.168.19.7  (seeds)
192.168.19.10
192.168.19.11 (seeds)

In each Scylla node, edit the scylla.yaml file

192.168.19.7

cluster_name: 'Xfs_Cluster'
seeds: "192.168.19.7,192.168.19.11"
endpoint_snitch: GossipingPropertyFileSnitch
rpc_address: "192.168.19.7"
listen_address: "192.168.19.7"

192.168.19.10

cluster_name: 'Xfs_Cluster'
seeds: "192.168.19.7,192.168.19.11"
endpoint_snitch: GossipingPropertyFileSnitch
rpc_address: "192.168.19.10"
listen_address: "192.168.19.10"

192.168.19.11

cluster_name: 'Xfs_Cluster'
seeds: "192.168.19.7,192.168.19.11"
endpoint_snitch: GossipingPropertyFileSnitch
rpc_address: "192.168.19.11"
listen_address: "192.168.19.11"

Start Scylla Nodes

Starts the nodes one by one.

$ systemctl start scylla-server

Using Scylla

Let’s create keyspace and table, try out.

cqlsh> CREATE KEYSPACE newdb WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
cqlsh> DESCRIBE KEYSPACES;

system_traces  system  newdb

cqlsh> DESCRIBE KEYSPACES;

system_traces  system  newdb

cqlsh> USE newdb
   ... ;
cqlsh:newdb> CREATE TABLE emp (id int PRIMARY KEY, name text, year text);
cqlsh:newdb> DESCRIBE tables;

emp

cqlsh:newdb> DESC emp;

CREATE TABLE newdb.emp (
    id int PRIMARY KEY,
    name text,
    year text
) WITH bloom_filter_fp_chance = 0.01
    AND caching = '{"keys":"ALL","rows_per_partition":"ALL"}'
    AND comment = ''
    AND compaction = {'class': 'SizeTieredCompactionStrategy'}
    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND dclocal_read_repair_chance = 0.1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99.0PERCENTILE';

cqlsh:newdb> INSERT INTO emp (id, name, year) VALUES (1, 'Krishna', '2017');
cqlsh:newdb> INSERT INTO emp (id, name, year) VALUES (2, 'Chandra', '2317');
cqlsh:newdb> SELECT * FROM emp;

 id | name    | year
----+---------+------
  1 | Krishna | 2017
  2 | Chandra | 2317

(2 rows)
cqlsh:newdb>

cqlsh:newdb> SELECT * FROM emp WHERE id=2;

 id | name    | year
----+---------+------
  2 | Chandra | 2317

(1 rows)
cqlsh:newdb> 

Data Export

Here’s the way to export data from Scylla to csv file.

cqlsh:newdb> SELECT * FROM emp;

 id | name      | year
----+-----------+------
  5 |    Namita | 2223
 10 |   Superna | 6623
 16 |    Olewaf | 3777
  1 |   Krishna | 2017
 19 |    Wqkdke | 2637
  8 |     Jeqya | 8937
  2 |   Chandra | 2317
 15 |    Luetea | 6993
  9 |    Namita | 2223
  3 | Prajapati | 2237

(10 rows)
cqlsh:newdb> COPY emp(id, name, year) TO 'emp.csv';
Using 2 child processes

Starting copy of newdb.emp with columns [id, name, year].
Processed: 10 rows; Rate:      19 rows/s; Avg. rate:      37 rows/s
10 rows exported to 1 files in 0.277 seconds.
cqlsh:newdb> exit
krishna@Ubuntu16:~$ cat emp.csv
16,Olewaf,3777
5,Namita,2223
15,Luetea,6993
8,Jeqya,8937
3,Prajapati,2237
1,Krishna,2017
19,Wqkdke,2637
10,Superna,6623
2,Chandra,2317
9,Namita,2223
krishna@Ubuntu16:~$

Data Import

Here’s the way to import data from csv file to Scylla.

cqlsh:newdb> SELECT * FROM emp;

 id | name      | year
----+-----------+------
  5 |    Namita | 2223
 10 |   Superna | 6623
 16 |    Olewaf | 3777
  1 |   Krishna | 2017
 19 |    Wqkdke | 2637
  8 |     Jeqya | 8937
  2 |   Chandra | 2317
 15 |    Luetea | 6993
  9 |    Namita | 2223
  3 | Prajapati | 2237

(10 rows)
cqlsh:newdb> TRUNCATE newdb.emp;
cqlsh:newdb> SELECT * FROM emp;

 id | name | year
----+------+------

(0 rows)
cqlsh:newdb> COPY emp(id, name, year) FROM 'emp.csv';
Using 2 child processes

Starting copy of newdb.emp with columns [id, name, year].
Processed: 10 rows; Rate:      15 rows/s; Avg. rate:      23 rows/s
10 rows imported from 1 files in 0.430 seconds (0 skipped).
cqlsh:newdb>
cqlsh:newdb> SELECT * FROM emp;

 id | name      | year
----+-----------+------
  5 |    Namita | 2223
 10 |   Superna | 6623
 16 |    Olewaf | 3777
  1 |   Krishna | 2017
 19 |    Wqkdke | 2637
  8 |     Jeqya | 8937
  2 |   Chandra | 2317
 15 |    Luetea | 6993
  9 |    Namita | 2223
  3 | Prajapati | 2237

(10 rows)
cqlsh:newdb>

Conclusion

We have successfully setup Scylla Cluster. Scylla is an alternative for Cassandra.
@Enjoy ….

Leave a Reply

Your email address will not be published. Required fields are marked *