MongoDB training report

严禁转载，使用请注明出处：http://alex2012-c.j.overblog.com/

有机会参加了两天的MongoDB training，写了一下报告（报告节选一部分）：

You will find here the notes that I’ve taken during the “Advanced MongoDB” training. First the overall impression I’ve had about Mongo as a DB solution for airport IT and then a bunch of problem solving scenarios that have been handled as exercises during this training and that could help you in a practical situation.
Overall Impression
MongoDB definitely sounds like a promising option for data storage. It is robust, easily scalable and convenient to use. But there are still some things that you would expect it to feature and that it doesn’t.

Advantages
 The fact that sharding is automatically done and supports compound keys is very good. It seems that without too much pain, you can have a distributed database and if you do it right, the performance can still be excellent with pretty much any quantity of data.
 The schema-less part can be a bit scary at first. Nothing is pre-defined and with the Bson format that even stores the nature of an object directly with its value, you can write anything anywhere. But if you assume that the developers who are going to use the DB are responsible and well informed, then it should be ok and even enjoyable after a while.
 A bit silly but still true : Jsons are more easily readable than SQL tables.

Defaults
I see one main difficulty with Mongo, and that is its inability to take care of the transactions by itself. Let’s take the shop example for instance and let’s suppose that you have a collection that represents the shop’s inventory and another one that stands for the shopping carts that are used by the customers.

Typical object in Inventory :
Product : {
Product_id: …
Description: ….
Available Quantity: …
}

Typical object in Shopping_Carts :
Shopping_Cart : {
Product : {
Product_id : …
Quantity :…
}
}

As per now, there is nothing on Mongo’s side that will give you the insurance that no product is removed from the inventory’s collection without having effectively being bought by a customer in the case where a disruption occurred while this customer is paying.
Of course it is manageable, like by adding a transitional state in the DB. Let’s focus on one customer’s check-out. We could start the process by adding the following field to the right product, right in the Inventory collection:
“checking-out”: {
“cart_id”: …
“quantity_bought_in_this_cart” : …
}
… and this one to the right Shopping_Cart in the Shopping_Carts collection:
“state”: “checking-out”
Then the customer would pay. And only once it’s done, before ending the process we could:
1) Delete the Shopping Cart from the Shopping Carts.
2) Decrease the “available quantity” in the Product object of the Inventory collection and clean up its temporary “checking-out” field.
On top of that, we could have asynchronous tasks that:
1) Delete the Shopping Carts that have not been used for a given amount of time
2) Clean the “checking-out” fields off the “product” objects in the inventory when they find no matching “checking-out” shopping cart.
This whole thing does the job but as you can see, it implies extra logic compared to what you could do with Oracle ( I believe ) and one could find dirty to actually write some fields in the DB that are not about actual information, but really just a work around to solve this transaction issue.
Transactions are on Mongo’s roadmap but it doesn’t seem to be easy to handle in the general case and nothing precise has been said about it. Anyway it’s surely not supposed to be included in the few next releases.

Scenarios and solutions

Backups and Recovery

Premise 1

MongoDB releases a new version. How do we upgrade our production systems with minimal downtime?
1) On a single node ( one node = one mongod instance )
a. Back up the data
b. Shutdown the mongod instance
c. Replace the existing bins by the latest ones (that you would have downloaded from mongodb.org/downloads
d. Restart the instance
2) Some details (you are in a putty command line or in a unix terminal)
a. How do I start a mongod instance?
i. mongod –fork –logpath /var/log/mongodb.log –dbpath /srv/mongodb/db this will record log output to /var/log/mongodb.log and store the data into /srv/mongodb/db
ii. if then you want to add this instance as a member of a replica set, use the replSet option ( for example “—replset rs0” means that this instance is usable as a member of the replicaset that has “rs0”as setname. In this case, you also need to note the host name and port information of this new host that is returned to you when you issue the command.
b. How do I shutdown a mongod instance ?
i. mongod –shutdown
c. How do I access the mongo shell ?
i. <path to bin>/mongo
d. How do I add up a created mongod instance as a member of a replica set ?
i. Connect to the replica set primary (if you don’t know which one it is, use db.isMaster() or rs.status() in the mongo shell )
ii. rs.add(“<hostname>:<port>”) That’s why you needed to note them when creating the mongod.
3) On a replica set (multiple nodes, as many mongod instances), that’s what you have in the general case and that’s where you can minimize downtime
a. Checkout the running nodes : ps aux | grep mongod
b. Typically, there are one primary ( to which go the reads and the writes by default by the way ), and one or several secondaries that insure redundancy. You could configure the replica set to let the reads target secondaries but you wouldn’t be sure of the exactness of the data you get as the secondaries are not exactly up to date. Let’s assume here that you have one primary and two secondaries.
c. Upgrade the version of one of your secondaries as explained above. Don’t forget to add up the new instance to the replica set again. Then do it for the other secondary.

d. You could just go on and do it for the primary. But the thing is if you kill the primary, the system will wait for 5 heartbeats ( 1 heartbeat = 2 seconds by default ) to elect one of the secondaries as the new primary. That means, you’ll have a 10 secs downtime. And there is a better solution : before killing the last mongod, you can manually trigger the election of a new primary by doing “ db.runCommand({replSetStepDown : 10})” on the primary in the mongo shell. Once it’s done, the old primary will be a secondary and one of the secondaries will be the primary. Then you can do the previous process on the last mongod without any downtime but the small gap you’ll have had during the election of the new primary

Premise 2

We have a new set of queries we want to run against the production system. But whenever we try to build the indexes for these queries, we experience a massive slowdown. How can we build indexes on production while minimizing the performance penalty?
It’s quite the same idea as what’s above. From the mongo shell, building an index is as easy as : db.records.ensureIndex( { userid: 1 } )
(this guy builds an index on the “userid” field of the collection “records”.
But doing this would use some RAM on the machine that does it of course and it might slow down the queries if it’s done on the primary. What you can do is doing it on the secondaries first, then trigger the election of one of them as primary as described previously, and then do it on the old primary.

Premise 3

Your primary has become unavailable because of a power surge. Fortunately, automatic failover kicked in and the system never went down. Unfortunately, you never set alerts to inform you when a node becomes unavailable, so weeks go by until you notice. When you bring it back up, the node does not restart cleanly. Find out why and fix it.
Here it was about the Oplog. What’s that? Remember that all the writes go to the primary? Well, the oplog is basically the buffer between primary and secondaries. When something is written in the primary, it also gets in the oplog and the secondaries pull data from it regularly. The oplog size is to be defined when the instance of the primary is created and of course the more updates you’re going to have by unit of time, the bigger the oplog needs to be and it should be large enough to contain at least a few hours of updates. In case you need to change the size of your oplog, it is possible and the procedure is described in mongo’s tutorials.

To come back to the exercise, in this scenario what happened is that the primary was down for a moment and so one of the secondaries became primary but the old primary was down long enough for the updates to be too numerous in-between to fit in the oplog. And in this case you have this node that cannot retrieve a part of the data simply because they have already been overwritten in the oplog when the node “woke up” and it might cause some malfunctions with it. To solve that we just killed it and made a new one.