Category Archives: Technology

Im somewhat of a geek, I like to write apps, programs, scripts and tinker with all sorts of random things. Whenever I build or do something I will post it here. I also run my own online tech business so any updates or changes will also make it in this section. In general im also pretty interested in many other things other then just IT related technology so if I find some cool articles or ready about something new and exciting you can be sure to find it here too.

Cara konfigurasi MongoDB replicaSet

Leave a reply

Quick Intro

So MongoDB has been the main database my team and I have developed our software platform on for the last 5 years. Its easy to learn and so far for us allows for a much faster development process.

We also found that as long as the right processes are put in place for apps that still require some sort of structure within the application, its a really nice DB to use if we still do want some relations between collections.

Now before we get into a spat about RDBMS vs NoSQL , this is just from the experience that I have had bringing my team up with complex software projects and the current technologies they are used to using since they are newer or younger software engineers.

To note, I personally spent a good portion of mys software engineering career using RDBMS’s but as of late I have also pretty much already moved away from RDBMS’s when I can help it and moved into the NoSQL world whole heartedly.

MongoDB Replica Sets

Now this post is not about MongoDB vs NoSQL or any sort of comparision but more about how to configure MongoDB as a replicaSet within a lab setting so that we can understand how it works and play with certain failover situations.

So quickly , what is a MongoDB replica Set configuration. MongoDB replicaset configurations are ways that we can provide a HA setting for our application (with HA meaning High availability).

HA configurations are used when the data needs to reside on many different nodes physically dis-joint and present in various different areas. Replica sets have a configuration where there is a primary node or master node which means the current active node for which read’s and writes to the database are made.

How to do the setup

Jadi sekarang yang penting, cara setup dan konfigurasi MongoDB buat replicaset. Please note, the assumptions are as follows:

3 nodes used for the replicaSet
MongoDB 4.2.8 being used (should be OK though for various versions)
Operating system is Centos 07
Assuming no DNS server to add the actual full DNS names

Step 1) Install MongoDB

Install MongoDB. The first step is to install the MongoDB database, as this is a Centos 07 installation we will update the repo to look at the MongoDB repo. To see a full instruction on how to do this we can refer to this document here by MongoDB.
But essentially the steps are:
- create the new repo at : /etc/yum.repos.d/mongodb-org-4.2.repo
- Add the repo details:

[mongodb-org-4.2]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/4.2/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.2.asc

Execute the yum install command as follows
- “yum install -y mongodb-org”

Step 2) Update the host file

Update the host files with the actual names of the hostname of the MongoDB servers so that we don’t have to configure them with IP addresses but with hostnames instead.
So assuming I have 3 servers I add them into the host file and give it the names

Step 3) Run the MongoDB installation

Lets look at how to run up the MongoDB processes. Note: If you are running the MongoDB as a lab setup you can run it as root , but as an installation for production please consider the proper security requirements for your setup to ensure that your database is safe and secure. With that , lets look at the setup.
Start the mongod process on each server :

Lets look at the command line parameters:

–bind_ip is the ip address that the mongod process will bind to
–port is the port that this process will listen on requests from
-dbpath is the path where all of the mongo data files will reside
–replSet is the name of the replica set

Note that in this case I am running the mongod with nohup , this ensures that mongod runs in the background. Now we repeat this process on each of the different servers which are servers

mongodb02.in.sidonesia.com on port 27017
mongodb03.in.sidonesia.com on port 27017

Step 4) Configure the replicaSet

After the MongoDB mongod processes have been run on each machine the next step is to create and connect all the nodes to create the replica set. The following steps will allow the process to happen

To get on this command line type in : mongo “mongodb://<hostname>:<port>”

The next step is to configure the replica set and initiate it , to do that we use the initiate command 🙂 as per below:
rs.initiate() # rs = replica set commands

After this is complete we add the nodes 2 and 3 into the replica set using the command rs.add(“<node_name>:<node_port>”)
The commands look as follows when adding the different nodes, also note, ensure that network connection is available between the nodes and that you an reach the port numbers of each host , you can do this by doing a telnet to the port of the other nodes which should yeild the following output

As you can see there is the ^] which means that connection is successful and the host mongodb01 is contactable and the ports are open for communication
Next run the replica set add commands so that you add each of the nodes into the cluster as per the commands below

As you can see above after adding each node the status of ok is set to 1 and the nodes have been added.

On the other machines , inspect the log and you should see output as follows where it shows that the MongoDB instance on the specific node has been added as part of the cluster

After the modes have added you can also check the status of the replica set by using the command rs.status(). This will show all the information regarding the replica set including the primary and secondary nodes
It will show some overall information and also information on each node stored in a JSON array.

This is the final step , now run the mongo client command from a separate machine that has access to the cluster and you should get a prompt.
The command is as follows: mongo “mongodb://mongodb01.in.db.sidonesia.com,mongodb02.in.db.sidonesia.com:27017,mongodb03.in.db.sidonesia.com:27017/?replicaSet=rs0”
This should get you into a mongo prompt and you can now test the cluster by stopping the nodes and continuing to do work

In summary that is how to get a 3 node cluster up , you can do various tests such as mongorestore a database and see that the data is replicated across all 3 nodes by inspecting the logs.

When doing a mongorestore you only need to send it to the primary node and the replication will occur. In the next post i will write a little about the architecture of the replica set and also show how to test the replica set to ensure that it is working as expected

Hope that helps 🙂

Mac Book Recovery ….

Leave a reply

After 4 days of sitting idle and not being used, the macbook has recovered. I don’t know why. I tried it two days later, it didn’t work so I thought it was definitely dead. Then I took it to the apple repair shop and bam ! The geniuses plugged my laptop into the power point, the adapter came on bright (something it hadn’t done for ages) and at that moment, I knew for a split second I was going to look like the biggest dimwit to ever walk into the apple store and complain that their laptop was not booting up 😛 These guys really are geniuses !

Well I didn’t care, dimwit or not, the laptop boots up and I can now get my environment of it and let this work horse lay to rest once and for all (Look at it, its melted, its dead, it needs to rest). So happy. I walked out and asked the apple genius what could have happened and he gave me some spiel about the fact that the power point might hot have had a constant current and the surge protection prevented the power from coming on properly and booting up my laptop.

Yeah … maybe … we are in Indonesia and these things can happen. I also read things online about a state the laptop can get into which is some discharge state where it lays dead for a few hours if you let the battery completely drain but from what I had read it would not do this for a period of days only hours. Could have been anything really and If i was less busy I would try to get to the bottom of this but not right now …

Anyhoo, all is well and now much of the stress that I had is lifted. Code might not work, there might be things failing and all sorts of problems happening but if you can’t fix it cause your tools are not working thats the real time to stress, what do you do when the thing you built is not working, twiddle your thumbs ?! *STRESS*, Other then that though, everything else and any other problem is fine … she’ll be right as we say in OZ 🙂 It’ll get fixed 😛

Have a good one guys, and jobsy, if your spirit came down to save my laptop i appreciate it … really, after being force to spend so much money just so I can develop on your amazing platform I really really appreciate it 🙂

OMG … Farking Macbook :(

Leave a reply

Its freaking comatose, its dead ! This is a freaking disaster. I have my Android coding environment on this thing and hadn’t had a chance to move the environment to my new macbook because I was so busy fixing up and finalising the changes on the iPhone. Code is safe, all in the repo but environment is not, got to set up all that crap and to be honest thats what I hate most about development. Development is fun, really really fun, BUT getting the stuff set up so you can start writing the code and start seeing your creation take life is the worst part of it cause really you just want to get down to the coding.

These instances are the moments when technology gets me really mad. I was hacking away on the new macbook and forgot that the power was plugged into the new one. I fell asleep on the table because I was so tired and it completely drain the power out of the old mac that I have 🙁 Now it doesn’t boot up ?! Is it so bad that the power drained ? If its so bad shouldn’t there be a warning ? Or maybe I missed it. Anyway this really really is the fail.

Lots of people with the same issue (battery drained no more boot) and some had this happen as a result of not so severe contact issues that just needed cleaning, others where magsafe connectors and mobo’s needed to be replaced, but why do these things make themselves obvious only after the battery is drained myst be a link in there somewhere ! I wanna throw this mac book in the bin and then incinerate that bin BUT right now its got my environment for android development and this is important. I just need it to boot up for 20 mins so I can copy the files, replicate the settings and be on my way on the new macbook.

Unbolt the hard disk you say, get the files of ?! I would (I opened the macbook) but the screws on these disks are a weird star shaped design that requires special screw drivers that for the most part no one would have (unless there was an apple genius or some sort of electrician possibly).

GRRRRRR …. OMG …..

*Takes car to Ratu plaza where apparently this is one place in Indonesia where they have apple geniuses there* ” This better work and better not cost an arm and a leg to fix” ….

e27 Ignite Event 2013

Leave a reply

Amidst the busy schedule of getting all the new software ready and primed the team went to the e27 conference today. It was important that we attended because it was all about the south east asian technology scene and how it is shaping up. It is important we get a clear view because we are about to launch our software within the region in the next month with our sales partners. As a result we wanted to get the news directly from the people involved within these regions to see how these regions are shaping up.

What we pretty much heard from all the speakers was as we had expected. The market is growing so large and fast yet still being ignored by many entrepreneurs due to much focus still on the traditional hubs such as the US. It was also pointed out that getting in early would be great strategy to adopt in this point in time.

There were experts on the panel from such regions as Indonesia, Thailand, Singapore and much discussion about regions such as malaysia & vietnam. It was a great event because it was confirmation that we are moving into markets that are about to explode and where the next wave of users seems to be coming from. Regions such as Indonesia have had massive growth in the use of smart phones and the use of Facebook and many of the other social media platforms has been spearheaded in the recent years by users coming from Indonesia and other nations within the region.

This means that as the population becomes more tech savvy at such a fast pace, they will consume internet and mobile technology at the same tremendous rate. The key is to get the technology and product right and to find the right channels into the market.

There were also some great pitches from various startups at the event and some of the highlights I thought were from mathspace, workdivision and soundgeko. I thought these where the cooler of the startups because being a tech-head I could imagine the technical expertise it would have taken to build these products and it really is amazing.

All in all it was a great day, its been a long time since I had to sit in a conference for so long but it was well worth it. Met many great people that I will certainly be contacting in future.

Only took a single photo though 😛

Technologies on “Blip A Deal”…

Leave a reply

I am what you would call a bit of a geek and love to play with different types of technology. For the last 8-9 years I have worked in corporates doing either development, design or architecture on either software projects, hardware projects or a combination of both whilst also in my spare time tinkering with all sorts of crazy stuff 😛

After years of working in larger companies about 1.5 years ago I (along with a friend of mine) decided to venture out and develop our own technology to hopefully make a living building and doing our own thing.

So it’s been one hell of a roller coaster ride but we are doing OK, it has its ups an downs but I wouldn’t have it any other way. One things for sure is it definitely beats the norm ! Also, as a result of building this business life has changed (in many ways) for the better in lots of unexpected ways.

We developed a site called BlipADeal found at http://www.blipadeal.com which is a platform for consumers around the world to find the best deals and coupons around. We also provide a large repository of deals and coupons for developers to access via our API’s. You can visit the site to get more information but in this particular post I mainly want to discuss the technologies that we used to build the service both on the mobile and web fronts.

The website

So the website was built using multiple different technologies which include, python, php, javascript, CSS, JQuery and HTML. Mainly the backend was developed using python frameworks and functions as an API so whenever the front end needs any new data it will make ajax requests and will receive only the specific information that it needs.

PHP is used to develop the front end dynamic pages. We could have used a python template library but decided that it would be best to use PHP so that updating designs on the fly would be much easier and would allow us to split the architecture much easier as traffic starts to grow (I might go through the architecture in another post later on).

JQuery and CSS is used for all the communications between the rendered HTML and the backend business logic and processing whilst also being used to render the presentation layer that interacts directly with the user. All data passed between the backend and the frontend is done through JSON objects. Its a nice little architecture and easily extendable to handle mobile requests also.

All of the software sits on top of the amazon cloud which runs EC2, S3, RDS and a whole host of other little services conveniently handled by amazon. Amazon is great and I think many founders can attest to how easy it has made their lives !

The Mobile Applications

In addition to the website we have also developed mobile applications so that users can easily receive their deals and vouchers on the go. So far we have supported mobile applications on both the main platforms, Android and iPhone. To save time we wanted to develop the application for both platforms using the Titanium (TA) framework but discovered that for android development there are very serious limitations.

We developed the iPhone application using Titanium pretty quickly and easily. All the code was developed using javascript and it was clean, easy and fast to test however once we came to developing the BlipADeal application for android we found the following limitations which actually where a pretty big deal.

Number 1 problem is that it would compile the whole javascript V8 engine into the APK binary file. So in essence this made the binary 20MB + where the binary should only be about 1 meg in total. This might not be a concern in places such as Australia and US where the mobile internet has pretty decent speeds but in some other asian regions (where we have a substancial user base) a 20MB file will simply be to large to download for everyone.

Number 2 problem is that its really difficult to have it supported on most of the devices and screen sizes, it just does not scale as well as if it was a native android app. This again is a big deal in Asia where many people purchase the more esoteric Android devices with weird screen sizes and models that Titanium complied applications simple wont run successfully on.

As a result of these pending problems we executed the mobile development in the following way, iPhone completed and done in Titanium whilst Android done natively using Android java.

The question always gets asked as to whether its better to develop natively or to use titanium. The answer to this is it really depends. As I described above, developing for Android using Titanium is a nightmare in my own personal experience. You have to develop many exceptions in your code and in the end to have a code base that works on both devices becomes really messy internally. This is simply not because of Titanium along but also because of the fact that there are some widgets that are not transferable between different devices.

If you want to develop for iPhone and want to get something up quick smart and you have existing skills in javascript using Titanium will exponentially increase your productivity. This means you could hook up and app in a week and have it on the store approved and ready. As far as my own experiences go with playing around with Titanium there has never been anything that I couldn’t do on the iPhone however I have never developed games and extremely complex layouts so there could be some limitations which I have not yet hit upon.

In terms of achieving a native looking app that functions well and is responsive, titanium can do this without any issues or problems on the iPhone platform. If you want to get something out as an MVP and don’t want to spend to much time on the MVP just to test the idea and want to do without spending more time developing the application natively, its the perfect option that in the intrim will buy you time to redevelop the application natively later.