GraphConnect 2020

Did you ever think you can setup your casual cluster to be self-healing and auto recoverable in the cloud? Would you like to know that your backups will restore without error and that your data is consistent every day? Come to our talk to learn more about running a stress free causal Neo4j cluster.


We will discuss:

• Restore testing / data consistency check
  • Confirming the backup was a success by executing a successful restore
  • Executing a consistency check on the restore
• Automated backups
  • Installed and scheduled to run on each node
  • Uses etcd cluster is as a locking mechanism to ensure only running on one node at a time in the cluster
• Autoscaling groups
  • Set to always have at least one server running
  • Extra ASG configured to help facilitate rolling upgrades
• CoreOS
  • Cloud Ignition
    • systemd units
    • Used to get secure keys and environment variables from S3 bucket.
    • Setup scripts
• Docker implementation
  • Use AWS ECR to store custom Neo4j docker image
• Ansible
  • Config management tool we used to configure all infrastructure.
• ELB endpoints
  • Uses native Neo4j calls to properly forward requests to a Leader or Follower
• ENI for persistent IP
  • Known private IPs for the cluster allows the use of a pre-ordained config file.
• How to select the right instance types
  • Hardware considerations
  • Memory is at least 2x database size - allows for growth - as well as some more for the OS.
• Gotchas
  • RAFT leader election issue with ephemeral ports
    • Add the correct port range to Security Group to allow for RAFT protocol
  • ENI CoreOS routing
    • Routing table rules needed for the configured private IPs
  • Unique constraints
    • What happens when you don’t add a unique constraint when adding a new Node Type that has an Id