MongoDB Replica Set 101

Introduction

A replica set is a group of mongod instances that maintain the same data set. A replica set contains several data bearing nodes and optionally one arbiter node. of the data bearing nodes, one and only one member is deemed the primary node, while the other nodes are deemed secondary nodes.

The primary node receives all write operations and records all changes to its data sets in its operation log, i.e. oplog

Secondaries replicate the primary's oplog and apply the operations to their data sets such that the secondaries' data sets reflect the primary's data set.

Replication provides redundancy and increases data availability. With multiple copies of data on different database servers, replication provides a level of fault tolerance against the loss of a single database server.

In some cases, replication can provide increased read capacity as clients can send read operations to different servers. Maintaining copies of data in different data centers can increase data locality and availability for distributed applications. You can also maintain additional copies for dedicated purposes, such as disaster recovery, reporting, or backup.

.

image.png

Basic setup

First of let’s create 3 EC2 instances on AWS and install mongoDB on it.

Make sure that mongodb service is running on all instances.

Screenshot from 2022-11-18 23-01-07.png

MongoDB primary server setup

Login to the server which you want to make primary member of the replica set

Step - 1 : Make the required changes for replica setup in MongoDB config file

vi /etc/mongodb.conf

Make the changes as follows

# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# Where and how to store data.
storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

# network interfaces
net:
  port: 27017
  bindIp: 0.0.0.0

# how the process runs
processManagement:
  timeZoneInfo: /usr/share/zoneinfo

#security:

#operationProfiling:

replication:
    replSetName: "myReplicaSet"

#sharding:

## Enterprise-Only Options:

#auditLog:

#snmp:

Note : Here i am setting the replica-set name and also binding the MongoDB with 0.0.0.0 to make it listen on all available interfaces.

After making above changes restart the MongoDB service.

sudo systemctl restart mongod

Step - 2 : Initializing Replica Set

Login to mongo shell and Check the status of replica set

#Login to mongo shell
mongo 
#check the status of replica-set 
rs.status()

You will get the following output

Screenshot from 2022-11-19 00-24-58.png It shows that still replica set has not been initialized.

Now let’s initialize the replica set

rs.initiate()

You will get the following output

initiate.png Note : As you can observer in image still the myReplicaSet has tag SECONDARY but once you logout and again re-login to the mongo shell then it will change to myReplicaSet:PRIMARY

Step - 3 : Adding secondary servers as replica-set member

Note : We will be using private IP address of the secondary servers while adding it as replica-set member.

rs.add("172.31.25.135")

You will get the following output

add.png

The same way add another server entry too as replica set member.

After adding server entries again check the status of replica-set. You will get the output similar to following -

ReplicaSet:PRIMARY> rs.status()
{
    "set" : "myReplicaSet",
    "date" : ISODate("2022-11-18T19:06:15.606Z"),
    "myState" : 1,
    "term" : NumberLong(1),
    "syncSourceHost" : "",
    "syncSourceId" : -1,
    "heartbeatIntervalMillis" : NumberLong(2000),
    "majorityVoteCount" : 2,
    "writeMajorityCount" : 2,
    "votingMembersCount" : 3,
    "writableVotingMembersCount" : 3,
    "optimes" : {
        "lastCommittedOpTime" : {
            "ts" : Timestamp(1668798366, 1),
            "t" : NumberLong(1)
        },
        "lastCommittedWallTime" : ISODate("2022-11-18T19:06:06.681Z"),
        "readConcernMajorityOpTime" : {
            "ts" : Timestamp(1668798366, 1),
            "t" : NumberLong(1)
        },
        "readConcernMajorityWallTime" : ISODate("2022-11-18T19:06:06.681Z"),
        "appliedOpTime" : {
            "ts" : Timestamp(1668798366, 1),
            "t" : NumberLong(1)
        },
        "durableOpTime" : {
            "ts" : Timestamp(1668798366, 1),
            "t" : NumberLong(1)
        },
        "lastAppliedWallTime" : ISODate("2022-11-18T19:06:06.681Z"),
        "lastDurableWallTime" : ISODate("2022-11-18T19:06:06.681Z")
    },
    "lastStableRecoveryTimestamp" : Timestamp(1668798366, 1),
    "electionCandidateMetrics" : {
        "lastElectionReason" : "electionTimeout",
        "lastElectionDate" : ISODate("2022-11-18T19:01:06.590Z"),
        "electionTerm" : NumberLong(1),
        "lastCommittedOpTimeAtElection" : {
            "ts" : Timestamp(0, 0),
            "t" : NumberLong(-1)
        },
        "lastSeenOpTimeAtElection" : {
            "ts" : Timestamp(1668798066, 1),
            "t" : NumberLong(-1)
        },
        "numVotesNeeded" : 1,
        "priorityAtElection" : 1,
        "electionTimeoutMillis" : NumberLong(10000),
        "newTermStartDate" : ISODate("2022-11-18T19:01:06.626Z"),
        "wMajorityWriteAvailabilityDate" : ISODate("2022-11-18T19:01:06.685Z")
    },
    "members" : [
        {
            "_id" : 0,
            "name" : "ip-172-31-90-148:27017",
            "health" : 1,
            "state" : 1,
            "stateStr" : "PRIMARY",
            "uptime" : 1318,
            "optime" : {
                "ts" : Timestamp(1668798366, 1),
                "t" : NumberLong(1)
            },
            "optimeDate" : ISODate("2022-11-18T19:06:06Z"),
            "lastAppliedWallTime" : ISODate("2022-11-18T19:06:06.681Z"),
            "lastDurableWallTime" : ISODate("2022-11-18T19:06:06.681Z"),
            "syncSourceHost" : "",
            "syncSourceId" : -1,
            "infoMessage" : "",
            "electionTime" : Timestamp(1668798066, 2),
            "electionDate" : ISODate("2022-11-18T19:01:06Z"),
            "configVersion" : 3,
            "configTerm" : 1,
            "self" : true,
            "lastHeartbeatMessage" : ""
        },
        {
            "_id" : 1,
            "name" : "172.31.25.135:27017",
            "health" : 1,
            "state" : 2,
            "stateStr" : "SECONDARY",
            "uptime" : 193,
            "optime" : {
                "ts" : Timestamp(1668798366, 1),
                "t" : NumberLong(1)
            },
            "optimeDurable" : {
                "ts" : Timestamp(1668798366, 1),
                "t" : NumberLong(1)
            },
            "optimeDate" : ISODate("2022-11-18T19:06:06Z"),
            "optimeDurableDate" : ISODate("2022-11-18T19:06:06Z"),
            "lastAppliedWallTime" : ISODate("2022-11-18T19:06:06.681Z"),
            "lastDurableWallTime" : ISODate("2022-11-18T19:06:06.681Z"),
            "lastHeartbeat" : ISODate("2022-11-18T19:06:14.505Z"),
            "lastHeartbeatRecv" : ISODate("2022-11-18T19:06:14.504Z"),
            "pingMs" : NumberLong(0),
            "lastHeartbeatMessage" : "",
            "syncSourceHost" : "ip-172-31-90-148:27017",
            "syncSourceId" : 0,
            "infoMessage" : "",
            "configVersion" : 3,
            "configTerm" : 1
        },
        {
            "_id" : 2,
            "name" : "172.31.26.100:27017",
            "health" : 1,
            "state" : 2,
            "stateStr" : "SECONDARY",
            "uptime" : 193,
            "optime" : {
                "ts" : Timestamp(1668798366, 1),
                "t" : NumberLong(1)
            },
            "optimeDurable" : {
                "ts" : Timestamp(1668798366, 1),
                "t" : NumberLong(1)
            },
            "optimeDate" : ISODate("2022-11-18T19:06:06Z"),
            "optimeDurableDate" : ISODate("2022-11-18T19:06:06Z"),
            "lastAppliedWallTime" : ISODate("2022-11-18T19:06:06.681Z"),
            "lastDurableWallTime" : ISODate("2022-11-18T19:06:06.681Z"),
            "lastHeartbeat" : ISODate("2022-11-18T19:06:14.505Z"),
            "lastHeartbeatRecv" : ISODate("2022-11-18T19:06:14.504Z"),
            "pingMs" : NumberLong(0),
            "lastHeartbeatMessage" : "",
            "syncSourceHost" : "ip-172-31-90-148:27017",
            "syncSourceId" : 0,
            "infoMessage" : "",
            "configVersion" : 3,
            "configTerm" : 1
        }

    ],
    "ok" : 1,
    "$clusterTime" : {
        "clusterTime" : Timestamp(1668798366, 1),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    },
    "operationTime" : Timestamp(1668798366, 1)
}

Here you can observe in Members section we have 3 entries in which 1st one has "stateStr" as PRIMARY but for other 2 entries have "stateStr" as SECONDARY .

Step - 3 : Creating database and adding some data

Let’s create a database name testDB and add some data.

#Creating databse
use testDB
#Adding one entry 
db.testcollection.insertOne({"Name":"Shubham"})

You will get the following output

ReplicaSet:PRIMARY> db.testcollection.insertOne({"Name":"Shubham"}) {
  acknowledged: true,
  insertedId: ObjectId("6377d95f06bc220aae79d510")
}
ReplicaSet:PRIMARY>

MongoDB secondary server setup

Now login to the secondary member of the replica-set

Now let’s try to fetch the data which we have added on our primary server.

#Login to mongo shell
mongo
#Fetch all available databses 
show dbs

You will get the following error

slaveError.png

Now Let’s allow read operations to run on secondary member.

#in mongo shell run the following command
rs.slaveOk()

Now try to fetch the data. You will get the output as following

Screenshot from 2022-11-19 02-00-18.png

Congrats 🎉 now we are able to read data on secondary member of replica set.

Connection string URI format

Standard connection string format -

Normally we connect to MongoDB with the connection string as follows -

mongodb://mongodb0.example.com:27017

Replica Set connection string format -

For replica set , we define the connection string as follows -

mongodb://mongodb0.example.com:27017,mongodb1.example.com:27017,mongodb2.example.com:27017/?replicaSet=myReplicaSet

Reference - MongoDB - Replication

Thanks for your valuable time. Please drop the feedback.

Did you find this article valuable?

Support Pranit Kumar by becoming a sponsor. Any amount is appreciated!