Fixing dependencies and autoremove packages in ubuntu

If you are a developer, then you might install various packages and remove them when work is over, or you get a better solution. However, most packages are not standalone, i.e they have dependencies on other packages, which apt-get  installs automatically. apt-get is really a very nice package manager. It internally keeps a list of what packages were installed automatically and which were manual. So, if you remove a package you manually installed, you might not want the other packages that were installed as a part of the dependency. 

    

     Here is a very nice answer from stackexchange:

Now apt implements an Auto-Installed state flag to keep track of these packages that were never installed explicitly. When you uninstall a package you can add the --auto-remove option to additionally remove any packages which have their Auto-Installed flag set and no longer have any packages which depend on it being there (a package may also be kept if another suggests or recommends it depending on the value of the APT::AutoRemove::RecommendsImportant and APT::AutoRemove::SuggestsImportant configuration options).
I would have a look at the list of packages and decide if they are worth keeping, sometimes packages which you may wan to keep are marked Auto-Installed by default. You can get information on what the various packages do by doing apt-cache show package_name. If you decide to keep some, you can use apt-mark manual followed by the names of the packages you want to keep.
Note that usually you would want to have library packages (most packages beginning with lib) marked as Auto-Installed since there are few reasons to have these packages installed on their own - other programs usually require other libraries to run, but they are little use on their own. Even if you are compiling software against the library to need the development package (ending in -dev) which depends on the library itself, so no need to explicitly install the library.
Also using aptitude, you can do aptitude unmarkauto from the command line or change within the curses interface. Within the package lists in the interface, all automatically installed packages have an A next to them. You can change this state by using m to mark an auto installed package as manual and M to mark as manual again (also l to open a search dialog and Enter to view package details)

This pretty much explains everything.

 Now, sometimes it happens that a package is installed without its dependencies, so obviously it does not work. apt-get keeps a list of the unmet dependencies.  You can resolve theses dependencies ,i.e install the packages required by

                                           sudo apt-get -f install

This will install the dependencies and your package will work as expected.


Now, as said earlier, how to get rid of packages that were installed as a part of dependency and no longer required? Here it is

                                          sudo apt-get autoremove

This removes the packages and frees up disk space.

Django. Django Admin Page and Phpmyadmin

I have been developing web apps for quite some time in Django. I would like to discuss here some things which might not have struck you while developing with Django.

Django comes with a inbuilt admin page, Once you register your model in admin.py , you are set to use the admin page for adding /deleting records in your database.However you might feel constrained by the functionality which this admin page provides, as I did. Since I come from a PHP background,  I badly missed phpmyadmin and its rich features. It feels quite strange to see  PHP and Python associated (it did for me).

    One thing is for sure, to run phpmyadmin, you need a server that supports PHP. So, first of all you need to know that you need to have your wamp/xampp/lamp server running in order to access the database. The default ip for wamp is 127.0.0.0 and that for django inbuilt server is 127.0.0.1 , so running both of them would never be a problem. Now you need to configure the settings.py file inside of your project with the database details like


                                             'ENGINE': 'django.db.backends.mysql'
and change the username,password etc. to access the database. Don't forget to run manage,py syncdb after this.  Now, how to make phpmyadmin notice this?

Head over to config.inc.php in htdocs directory of your wamp installation and provide the connection details to the django built in server in the manner it has been done for the apache server above . Remember to put the $i++ before the configuration details.


Now, restart apache and go to phpmyadmin. You will get an option to select beween the 2 servers. Choose 127.0.0.1 (django) and you will find the database sitting right there.

The bad thing about this is:
You cannot modify the database structure. You can only add/update/delete records from the database. There must be a tool for overriding this, so I will update later!!

Handling change in usernames, in a better way!

Some websites that have account for users , give an option to change their username. It appears to be a trivial task. I have implemented this functionality in 2 of my projects . The functionality i used was to ask a LOGGED IN user for the new username and would ask him to reenter the password. This is a very simple approach and works well for small applications that do not have very much functionality in terms of databases.

     

     I recently changed my username on Github and was wondering what procedure they followed. I think it is highly improbable that they use the usernames that we see internally in databases too. What I mean is that suppose Mr. XYZ does a commit in his repo in Gihub (XYZ is the username), so in their database that stores the history of commits, XYZ is not stored, rather a hash( for more security) or a simple unique number is stored. Their would be a user table where each such number or hash uniquely determines a user and does not change anytime, no matter how many times the user changes his username. Whenever some information is displayed that should identify "who ? ", they would simply fetch the username associated with the unique number or hash (of course they could store this number in a session variable to avoid querying the same thing). This gives the advantage is that you don't have to traverse through your entire database and substitute the old username with the new one. You simply change the username in the user table and the unique identifier(the hash or the number) remains the same. It would greatly bring down computations required for trivial task like this. 

Processing Feedbacks

It's a day before the endsemester exams and I can't help thinking about how the ERP team evaluates my feedback about professors. 10,000 + students. One obvious way which I can think of is- It gets a overall rating of each teacher. If the overall rating is good enough, I don't think they worry about the suggestion. If there is a poor rating, they would manually take some random suggestions (if they take at all).

Improper and inefficient I would say (if at all this is the process).

I am thinking of using Python's Natural Language Toolkit for this purpose. I will look for words like "good" "bad" "should" and get their context like "good at *" "bad at *" "should do *". Similar suggestions (a group of students giving similar suggestions) would be forwarded to the teacher to improve the classroom experience. Further updates after the endsem.

Ordering your Notes

You are a student and did not make notes properly for a particular subject? And you took photos of your friend's notes to study? Then this post is for you.

I too had such a situation once. One thing to highlight is the naming convention by most devices. Most phones and cameras name the shooted images as yyyymmdd_hhmmss .

Obviously, this naming convention makes sure that no two photos have the same name .(I have not tested what happens when you deliberately change your device time to an earlier time which corresponds to an already present image on device, though it is very difficult to remain accurate to the second.)

Python to the rescue.

My method is not efficient but it is quick and "quick" is what i wanted. So,  i sorted out the names of the images on the basis of when they were clicked (to be more accurate, time when they were clicked) and then renamed them with integers starting from 1 .

This way you can easily remember what page you were on when you last left.

My code is available at https://github.com/light94/rename-notes


UPDATE:
I have updated my code to rotate the images by 90 degree so that everytime a new image comes, we don't have to manually rotate it.

Note: One important thing that I learnt from this was that python's os.listdir() generates the list randomly if no parameters are passed to it. 

Fixing Permissions of htdocs in Ubuntu

If you are using xampp, you might already know that the default location of your projects will be /opt/lampp/htdocs . However you cannot edit contents of this folder without root permissions. However, using the root account for trivial things like editing files is a very bad idea since you can do harmful changes to your system unknowingly.

There is a very detailed and beautiful description of how to avoid this in this blogpost.

The idea is, we need to change the ownership of the folder to something other than the root. No more sudo for triviality.!! Go ahead and read that article.

UPDATE: The -R flag to chown applies command to all the subdirectories( recursively) as well.

Browser and the Event Loop

This is an extension of my last post and I wrote it after watching Philip Robert's speech.

A browser is constrained by what you are doing with the Javascript. The browser would like to repaint /render the screen 60 frames per millisec, but it can't render if there is code in the call stack. The render is like a callback with the difference that it gets a priority among callbacks. That is, if the stack is empty and there the render is in queue, it will be executed despite the fact that any other callback is before it.

While render is blocked, you cannot select text on screen, click on a button and see the response etc. That is why we should not put slow code on the stack . We should not block the event loop.

V8, Javascript, Event Loop

This is what i understood form Philip Robert's speech at JSConf EU 2014
V8 is Google's Open Source Javascript Engine. It is written in C++ and is used in Google Chrome.

The Big Picture,

i.e Java runtime, webapi(extra things which the browser provides), event loop and the callback Queue.

Javascript is a single threaded runtime, i.e it has a single call stack (it can execute only one piece of code at a time).

(Note: A call stack is basically a data structure which records where in the program we are.)

Blocking - Essentially, it means code that is slow and therefore preventing execution of statements after it. For example a while loop, which runs for say 100 times. Until the while loop is over , statements after it won't be executed. The browser cannot render anything until there is some process in the call stack.

The solution to blocking - asynchronus callbacks.

Now, the browser is more than the runtime as shown in the picture. It has other components like Web API, event loop which help in concurrency,i.e , executing more than one instruction at a time.

Consider this piece of code 

If we look at the call stack, the callback queue and the event loop together, we see the following picture initially.

When we start the script,we find the picture to be this

When console.log('Hi') is executed,

Now, as we saw before, setTimeOut is a API provided by the browser, so setTimeOut(cb) sets off the timer and is then handled by the browser. So, just after the statement is executed, we find

that is, the setTimeOut is not there in the callstack any more and thus the next console.log() statement is executed and clears off after which the main() clears off (pops off).The browser is handling the setTimeOut. The WebAPI cannot simply push the command back to stack.

Here comes the role of the Event Loop . It looks at the stack and at the queue. If the stack is empty, it pushes the first thing in the queue to the stack where it gets executed.

Note: We use a timer of 0 to setTimeOut when we do not want the statement to be executed until the stack is clear.

setTimeOut is not a guaranteed time of execution. It is the minimum time of execution.

Cursor Blinks in Linux ,Noun Not Verb

Of late, I had been troubled by the distinct behavior of the cursor in my Ubuntu setup. Symptoms?

1 .The cursor blinks for sometime and you believe that everything is going fine.
2. After a timeout, the cursor stops blinking and you begin to wonder and worry.


To be more precise here is what the original bug reporter said,(if you call that a bug)

When the cursor stops blinking, you start to wonder if something is wrong.

The point is that the cursor should never stop blinking in an active terminal window

Well, this is bug #838381.  

This is a general feeling. However, after coming to this page, I have now stopped worrying.

Let me sum up what the article said. 

The stopping of blinking of the cursor is not a bug rather a deliberately introduced feature to save energy.

So, even though the cursor is not blinking, your code is working just as it should, behind the scenes.Period.

Using Virtualenv in Python

Virtualenv is a tool to create isolated Python environments. This may be useful if :

• You need to test some module
• You are doing a project that makes use of a different version of  a package that is already installed.

So, I am installing pygame on my ubunutu and don't want this installation to be system-wide. Therefore I decided to use virtualenv.

To make a virtualenv, 

1. Go to the parent directory and open terminal in that directory.
2. Type virtualenv --no-site-packages <<a folder name>>  
3. Then cd to the folder and type source bin/activate

This will create a virtual environment where you can install new python modules without the fear of it affecting the main python installation.

When your work is over, you can close the virtualenv using deactivate.

Further reading http://www.davidfischer.name/2010/04/why-you-should-be-using-pip-and-virtualenv/

Creating custom bash scripts

Often we across activities where the same set of actions need to be performed. These can be automated using bash scripts in Linux.

To make a bash script:
1) Begin the script with a shebang #!/bin/bash
2) type in the custom commands. Use $1, $2 etc. for passing in arguments to the script
3)Give it a name and save it in /bin   (Create a folder in home directory named bin)
4)Next , to make the file executable, type chmod +x <filename>

 Now you are ready to use the power of automation in linux. Have a look at https://github.com/light94/myShellScripts .

Hacking with Nautilius actions

I have been using Sublime Text for quite some time now and am an absolute fan of it. There is a very nice feature in it - open a folder . What it does is  that it gives you a list of all files in the folder in the package explorer pane. This makes it easy to select any file in the folder or editing, closing and reopening.

This is indeed very useful. But i thought of adding an alternative. Suppose, you are browsing through your projects and you stumble upon one of them and decide to work on it. It is painful(for me atleast) to open sublime text and then open in folder. What if there is an option present while browsing to open the folder in sublime ? Sounds cool.

So this link came to the rescue http://www.howtogeek.com/116807/how-to-easily-add-custom-right-click-options-to-ubuntus-file-manager/

Basically you need a tool called Nautilus Actions that you need to install by sudo apt-get install nautilus-actions. You then need to launch it and add a new action. Put a suitable name in Context Label (this is what will appear in the right click).  Then go to command tab and write subl in path and -a %F in parameters. subl is actually the command to launch sublime text from terminal . The -a argument adds folders to the current window and %F passes the name of the current dirctory to the command.

Then close nautilus and reopen it . Now when you right click on any folder, you will get the option to open it in sublime.

Onepiece Fans, Behold

After the script to download Naruto Shippuden episodes(https://github.com/light94/naruto), here(https://github.com/light94/onepiece) is the script to download Onepiece episodes. This is more customized than the previous.

Both websites offered almost the same structure and therefore the procedure is almost the same.

Facebook's Profile Picture Privacy Update

This is an update from an earlier post (http://justanotherlearner.blogspot.in/2014/08/facebook-bug-maybe-maybe-not.html) where I had misunderstood a feature of FB and pointed it out to be a bug.
Today, I came across the facebook privacy page (https://www.facebook.com/about/privacy/your-info#public-info)  where it is written that

"Your name, profile pictures, cover photos, gender, networks, username and User ID are treated just like information you choose to make public"

So, it is absolutely fine if you are able to download the profile pictures of others. However, FB still provides the option of keeping your profile picture private . There is certainly a discrepancy in information and it should be resolved.

Saving Passwords in Git

For git version >= 1.7.10
Entering the username and password everytime you do a git push is a waste of time. The following commands will save your credentials for a defined time period in memory.

To store your username and password, use the following commands

git config --global credential.helper cache

By default git stores password for 15 minutes. You can change this by

git config --global credential.helper 'cache --timeout=3600'

(If you want a timeout of 1 hour)

"Gotta" Catch'em All exceptions

So, I was recently doing a webscraping project. I had a database of people and I had to lookup and confirm their identity. Identity in the sense that they had either studied or taught in my college. The task can be done manually for a small set, but I had 7000 records!!

So I used xgoogle for this purpose. It is a Python Library for Google Search.

Now, after having completed the task, I can name a minimum 7 exceptions that can occur when you automate web lookups. They are :

SearchError (class in xgoogle that handles exceptions raised by google during searches)

HTTPError  (generally raised when there is some error in authentication, like trying to access a secured page of an organization, or when the url is http when it really should be https)

URLError (being a subclass of OSError, this error is related to system related problems like input/output . This includes error in fetching data from a url)

SocketError (Raised when there is some issue with the server eg. overload, throttle)

ValueError (Raised when the required datatype conversion is not permitted)

IncompleteRead (When the server closes the connection before the chunk of data has benn fully retireved.)

BadStatusLine (Raised when we request a webpage, and the server responds with a code that Python cannot understand)

Well this is not the complete list.

I had to automate the task in such a way  that I start the code before going to bed and I get it completed when I wake up. So, I decided to catch all :

try:

    my code

except KeyBoardInterrupt:

    raise

except:

 #handles all exceptions except KeyBoardInterrupt

This piece of code will ensure that unless you manually try to end your code, it will not stop.

Testing your android app on your phone

This is something straight from the source(Google's android resources) and very important, How to test your android app on your phone?

Enable USB debugging on your device.

On most devices running Android 3.2 or older, you can find the option under Settings > Applications > Development.

On Android 4.0 and newer, it's in Settings > Developer options.

Note: On Android 4.2 and newer, Developer options is hidden by default. To make it available, go to Settings > About phone and tap Build number seven times. Return to the previous screen to find Developer options.