Archive

Archive for the ‘Uncategorized’ Category

Infographic: The Big Vs of Big Data – Volume, Velocity and Variety

October 24, 2013 Leave a comment

Infographic: The Big Vs of Big Data - Volume, Velocity and Variety

Infographic: The Big Vs of Big Data – Volume, Velocity and Variety

Pinterest Drives More Traffic to Publishers Than Twitter

October 22, 2013 Leave a comment

Pinterest Drives More Traffic to Publishers Than Twitter

Pinterest Drives More Traffic to Publishers Than Twitter

Vine is the Fastest-Growing App in the World

October 22, 2013 Leave a comment

Vine is the Fastest-Growing App in the World

Vine is the Fastest-Growing App in the World

Infographics: Worldwide Nuclear Warheads distribution from 1945 – 2013

September 15, 2013 Leave a comment

Nuclear Countries along with the year of their appearance:

Image

 

Highest count of nuclear warheads for each nuclear country in specific year:

 

 

Image

 

Nuclear countries along with their Intercontinental ballistic missile range:

 

 

 

Image

 

USA Nuclear Warhead Stock per year:

Image

USSR/Russia Nuclear Warhead Stock per year:

Image

UK Nuclear Warhead Stock per year:

Image

France Nuclear Warhead Stock per year:

Image

China Nuclear Warhead Stock per year:

Image

Israel Nuclear Warhead Stock per year:

Image

India Nuclear Warhead Stock per year:

Image

Pakistan Nuclear Warhead Stock per year:

Image

 

Source: Bulletin of the Atomic Scientists

Source: http://en.wikipedia.org/wiki/Intercontinental_ballistic_missile

 

 

Categories: Uncategorized

Data Visualization and Info-Graphics Resources

September 3, 2013 Leave a comment

Image

 

Data Visualization and Info-Graphics:

Portfolio and collective Work:

 Other Resources:

Software Tools

 

Source: http://www.quora.com/Where-are-some-good-sources-for-data-visualization#

 

 

Categories: Uncategorized Tags:

Foundation steps to become a leader

August 28, 2013 1 comment

Recently I went through a literature which was given to me during a leadership training at my previous work. I wanted to live through the training again and decided to follow it again completely. The content was great and it was mainly conveying 3 main things to become a successful leader:

  1. Translating individual potential to team potential and individual strength to team strength
    1. This is done by showing trust in team and confidence
      1. Winning trust is not an easy thing and it is done be being honest
      2. Confidence in team is created by encouraging them to do the exemplary things by showing the path, not by doing it yourself
  2. Searching hidden potential in team and bringing it the limelight
    1. This is done by preparing and promoting team member for bigger task
      1. This is achieved by seeing big and looking for the big picture
      2. After that showing the exact same picture to everyone
      3. Finally bringing everyone to follow the same objective
  3. Do less and achieve BIG
    1. This is hardest to achieve because a leader want to see the results at his scale and to match then, he or she starts doing everything by himself and it is the end of leadership.
      1. A leader is nothing if his or her team does not see himself or herself a leader and you can not force someone to follow your lead.
        1. This is achieved by mastering #1 and #2 above.

A leader is consider to be like the “paddle” for the boat. The “paddle” seems to look very small comparative to the “boat” however it is “paddle” who gives the direction to boat and help reaching to its destination . Everyone sees the “boat” however no one see the “paddle”. Now these days leaders are like “flag” on the boat who are there to show off and get the credit from the “paddle”.

Adventure with Postgresql on Ubuntu


Setting postgresql to not start during startup:

To enable or disable postgresql start at machine startup edit start.conf located @ /etc/postgresql/9.2/main/ folder and add proper configuration auto |  manual | disabled 

$sudo vi /etc/postgresql/9.2/main/start.conf

# Automatic startup configuration
# auto: automatically start/stop the cluster in the init script
# manual: do not start/stop in init scripts, but allow manual startup with
# pg_ctlcluster
# disabled: do not allow manual startup with pg_ctlcluster (this can be easily
# circumvented and is only meant to be a small protection for
# accidents).

auto

postgres database server Startup or Shutdown:

postgresql database server can start directly as service from services tool i.e. “sudo services postgresql start|stop|*” or using a wrapper named pg_ctl, as below: 

$sudo services postgresql start

$ /usr/lib/postgresql/9.2/bin/pg_ctl 

If postgresql is start through wrapper with specific catalog, then the status display different results as below:   

$ sudo service postgresql status
9.2/main (port 5432): down

$ /usr/lib/postgresql/9.2/bin/pg_ctl status -D /home/hadoopuser/hadoopuser_DATA/pgdb -l /home/hadoopuser/hadoopuser_DATA/logs/pg.log -w 

pg_ctl: server is running (PID: 3381)
/usr/lib/postgresql/9.2/bin/postgres “-D” “/home/hadoopuser/hadoopuser_DATA/pgdb”
hadoopuser@HADOOP_CLUSTER:~$

Handling portgresql error “could not create lock file “/var/run/postgresql/.s.PGSQL.5432.lock”: Permission denied”

Sometimes starting postgresql server returns the following error:

WARNING: could not create listen socket for “*”
FATAL: could not create any TCP/IP sockets
FATAL: could not create lock file “/var/run/postgresql/.s.PGSQL.5432.lock”: Permission denied

The problem is that in ubuntu postgresql is running and during installation hadoopuser is starting it and because of ownership issue hadoopuser can not use it and user see the lock issue as shown in above logs:

 

hadoopuser@hadoopserver:~$ ls -l /var/run/
drwxrwsr-x 2 postgres postgres 40 May 28 22:16 postgresql

hadoopuser@hadoopserver:~$ sudo chmod 777 /var/run/postgresql

hadoopuser@hadoopserver:/usr/local/hadoopuser/current$ ls -l /var/run/
drwxrwsrwx 2 postgres postgres 80 May 28 22:32 postgresql

 

Handling postgresql error “Is another postmaster already running on port 5432″

Sometimes starting postgresql server returns the following error:

HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
LOG: could not bind IPv6 socket: Address already in use
HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
WARNING: could not create listen socket for “*”
FATAL: could not create any TCP/IP sockets

To solve this problem please first check if postgresql is running as below:

hadoopuser@hadoopserver:~$ ps auxwww | grep postg
hadoopuser 1805 0.0 0.1 137520 10212 pts/0 S 22:25 0:00 /usr/lib/postgresql/9.2/bin/postgres -D /home/hadoopuser/hadoopuser_DATA/pgdb
hadoopuser 1807 0.0 0.0 137520 1492 ? Ss 22:25 0:00 postgres: checkpointer process
hadoopuser 1808 0.0 0.0 137520 1732 ? Ss 22:25 0:00 postgres: writer process
hadoopuser 1809 0.0 0.0 137520 1492 ? Ss 22:25 0:00 postgres: wal writer process
hadoopuser 1810 0.0 0.0 138300 2776 ? Ss 22:25 0:00 postgres: autovacuum launcher process
hadoopuser 1811 0.0 0.0 97208 1572 ? Ss 22:25 0:00 postgres: stats collector process
hadoopuser 2165 0.0 0.0 8104 924 pts/0 S+ 22:32 0:00 grep –color=auto postg

Stop the postgresql service as below:

hadoopuser@hadoopserver:~$ sudo service postgresql stop
* Stopping PostgreSQL 9.2 database server [ OK ]

Note: Above command does not guarantee to stop services if the database server was started in other user context and actively running with an open database. 

or kill the postgresql process as 

hadoopuser@hadoopserver:~$ killall postg

 

Categories: Uncategorized

Upgrading Pycrypto using pip in Ubuntu


Here are the steps to upgrade pycrypto library in ubuntu machine:

Step 1: check pycrypto version

ubuntu@ip-***:~$ pip show pycrypto

Name: pycrypto
Version: 2.4.1
Location: /usr/local/lib/python2.7/dist-packages
Requires:

Note: If you dont have pip working try installing 

$ sudo apt-get install python-devel

$ easy_install pip

 

Step 2: upgrade pycrypto using pip

ubuntu@ip-10-254-71-179:~$ pip install –upgrade pycrypto

Downloading/unpacking pycrypto from https://pypi.python.org/packages/source/p/pycrypto/pycrypto-2.6.tar.gz#md5=88dad0a270d1fe83a39e0467a66a22bb
Downloading pycrypto-2.6.tar.gz (443kB): 443kB downloaded
Running setup.py egg_info for package pycrypto

Installing collected packages: pycrypto
Found existing installation: pycrypto 2.4.1
Uninstalling pycrypto:

…..

Successfully installed pycrypto
Cleaning up…

 

Step 3: Verifying the upgrade

ubuntu@ip-10-254-71-179:~$ pip show pycrypto

Name: pycrypto
Version: 2.6
Location: /usr/local/lib/python2.7/dist-packages
Requires:

 

 

Amazon EC2 Security Group (Firewall) settings for Hadoop Cluster


When setting Hadoop cluster in Amazon EC2 you would need to configure proper security settings (firewall) so you can access Hadoop cluster directly. Following are the settings for Cloudera CDH4 Hadoop distribution on EC2:

Image

 

 

Port 22 for SSH, Port 7180/82 for CDH Manager, 7432 for PSQL and 8888 for Hue and finally Port 50000-50100 for Hadoop JT and HDFS.

 

Categories: Uncategorized Tags: , , , ,
Follow

Get every new post delivered to your Inbox.

Join 37 other followers

%d bloggers like this: