Adding/Removing stuff from start-up

Its a wonder how frequently we need to add stuff to the start-up and yet we prefer the hard-ass manual way. You can lose hours if you forget to do one small thing (eg. one of my builds, which takes 2 hours on an average, failed merely because I forgot to start httpd.) Thats when I considered this start-up thing seriously.

1) Adding/Removing services:
In very (very) loose terms, services comprise of everything under /etc/init.d directory. (Files under this directory are scripts that manage starting, stopping, restarting and status reporting of the services). The 'chkconfig' command is used to Add/remove these services from the start-up. These can be done per-runlevel too. (Just pass a suitable '--level' option)
eg. To start httpd on startup. Just type
>> chkconfig httpd on
(To remove httpd from startup, replace 'on' with 'off')
(Syntax ==>> chkconfig {service-name} on/off)


(To add a script to be managed by chkconfig, just copy the script to /etc/init.d directory and run
>> chkconfig --add {name}
This creates links in /etc/rc[0-6].d directories. Scripts in these directories are run each time the run-level is switched (provided that the service isn't already running))

Type in 'chkconfig --list' to get a nice summary of the status of services at each runlevel.


2) Initializing environment variables at start-up:
Just add an "export name=value" command to ~/bash_profile file. (Create the file if it isn't already there). Note that this would be a per-user initialization.
To initialize an environment variable globally, add an entry to the /etc/profile file.


3) Initializing aliases:
Add a "alias name=value" to ~/bashrc file. Create the file if it isn't already present.
/etc/bashrc file if you need global changes.


4) Random stuff in start-up:
What if you want to run a random command at start-up? (eg. Flushing ip-tables). In fedora you can add the commands to /etc/rc.d/rc.local file. But, the file is distro dependant. Most probably, you'll find the appropriate file in /etc/rc.d directory.

Hope it helps!

Patching....

Before we delve into applying patches, let us have a look at a sample patch file.


--- old_test 2008-12-07 15:43:27.000000000 +0530
+++ new_test 2008-12-07 15:43:39.000000000 +0530
@@ -1,5 +1,6 @@
This
is
+also
a
test
file


The first line gives details about the original file and the second line about the modified file.
Then "@@ -1,5 +1,6 @@" marks the beginning of the hunk. This line tells the patch command where, in the file-to-be-patched should it find the specified change.

The line prefixed by "+" is the line that was added in the modified version. Other lines constitute the context.

Now why is context important?
Here's why. Consider this scenario. You work on this amazing new idea on a file called blah.c. At the same time, another of your friend modifies the same file to implement his crazy new idea. So, would your patch not apply to your friend's version of blah.c?
Chances are, it will!.. because patch tries to find the same "context" in the neighbourhood of the original line numbers. If it does, patch applies successfully!... Now this is what makes patches a good idea!.. DISTRIBUTED DEVELOPMENT!

So, coming back to the patch command,
>> patch -pNUM < patch-file

Note that patch reads the patch file from the standard input. So, you have to redirect the standard input to the patch file.

the -p option is used to specify number of components to strip.
Here's a line from a patch file. Lets take it up as an example to understand the -p option
diff --git a/wlan/driver/linux/some-name.c b/wlan/driver/linux/some-name.c

the -p option works on the path "a/wlan/driver/linux/some-name.c".
with -p1 : it strips one element from the LHS of the path. Thus, it will try to find "wlan" directory in the current directory.

with, -p2 : patch will strip the first 2 components. i.e. "a" and "wlan" and it will try to find "driver" directory in the current directory.

and so on.
If you don't give a -p option, -p1 is used by default.


Unapplying patches:
It is possible that you applied a wrong patch to your code. Or you find out after applying the patch that the damn thing is not going to work. Or there is a regression. Whatever. The point is, you want to revert the patch.

Just pass a "-R" option to patch and it will try to unapply the patch. Although, do read the man page carefully before using this option.


Random notes on the patch command:
1) Use the strip-level option (-p) properly. If patch command doesn't find the file to be patched, chances are that you have specified an improper strip level option. (In that case, patch becomes interactive and asks you the path of the file to be patched. Most of the times, it means that you need to correct the strip-level option. so, exit with ctrl+c and give the correct strip level)

2) In case, one of the hunk fails (Which is the case when, the original file is modified by someone else prior to applying the patch and there is a conflict), the hunk that failed is saved in the "filename.rej" file. You need to apply that change manually. Just open the .rej file, see which hunk failed and make the changes manually. Also, the original version of the file is saved in filename.orig, just in-case you changed your mind or you are just too lazy to do the hard-work.

3) A patch is a defacto standard to give out changes like bug-fixes, minor feature additions, security patches etc. Patches are extensively used when a team is working on the same workspace (which is almost always the case). Revision control softwares like git, SVN, CVS, perforce, etc all support patches extensively in all their operations.
Patches are something you really want to know!

Hope it helped!

Diff and Patch

Patches show the difference between two versions of the same file.
A patch shows which lines were newly added and which lines were removed. What makes a patch very useful is that a patch can be applied to the old version of the file to obtain a new version.
i.e. (old version) X (Patch) = (new version)

This fact makes it easy to manage a community software where a software is open-source and anyone can submit changes. Trust me on this, without patches we would be nowhere.
For eg, Andrew Morton (Linux kernel maintainer) gets hundreds of patches everyday. In the absence of patches, he would have to integrate each and every change manually which would be a VERY time consuming and an inaccurate process. Patches automise this job.

"Diff" is used to create patches and "patch" is used to apply it.


Diff:
syntax
>> diff [options] [original version] [modified version]
Generally, I used the diff command with "-du" options.
eg. diff -du old_test.c new_test.c > test.patch

Note that diff produces output on the stdout and you'll have to redirect it to a file.
Options:
-d : Use an algorithm which will produce smaller set of differences
-u : Use unified output format
(-u causes newly added lines to be prefixed by "+" and removed lines prefixed by "-". Otherwise, other prefixes are used which people generally dislike :)

-r : recursive.
Now this is a beauty. You can find differences between all files under a directory. So, if you have changed multiple files, use this option to generate only ONE patch and _not_ a patch for each file separately.

-N : Include new files
With -r, it is possible that you might have added a new file in the newer version of the software which wasn't there in the old version. Diff by default ignores such files. Use this option to change this default behaviour.

Random notes on patches:
1) In patch files, you will find something called hunks. Hunks are a set of changes alongwith some context. By default, 5 lines above the change and 5 lines below the change constitute the context. View a patch file in emacs/vim and you'll know what I am talking about. Context is a very important part of the patch file. I'll come back to it when I talk about applying patches.

2) The beauty of patch files is that you can actually MODIFY them. Well, it won't seem to be a big deal if you havn't used patch much. But, it is a quite handy trick. You can remove entire hunks and the patch is still valid!
So, play around with it a bit.

3) Many people and inherently afraid of patches. Trust me they'll have a very hard time when they work in a group on a single project. Patches are a great way to review changes that have been made or pass them on to others. Without patches, I cannot concieve a way I'll pass on my changes to others!
Patches are very easy to use. Just try and get comfortable with them. Ask around if you think you are stuck.

Now that we know how to generate patches, it is time to move ahead and learn how to apply these patches.
But, I think this post is long enough. So, I'll write about it in the next post.

(Do leave a comment and let me know if I need to add/remove/modify anything to make the post lucid and easily understandable yet comprehensive)

How to browse source-code?

99% of your life is wasted is studying code written by someone else. And studying this code can be an awfully irritating task. (Although, studying codes written by TE/BE students can be a really pleasing experience. You can just lambaste them on every possible line! It is fun. Trust me. Simply log-on to the lab-server, access someone else's code. Don't worry about the permissions. Seems that PICT's lab assistants believe in open source. So, get someone else's code and vent your anger through comments. Don't forget to save the file :D)

Anyway, getting back to the point. There are few tools available which can make the task a bit simpler.
1) Etags
2) Cscope
3) Doxygen

My good friend Vedang has written enough about them. Here are the links to Etags and Cscope posts.

Personally, I think Cscope is just amazing. You can find functions which call this particular function and there is a regular expression search too. Etags is quick but primitive. I generally use Etags for simple browsing and Cscope only when Etags cannot help!

Tools aside, you also need to have a strategy for browsing source code. You cannot simply dig into the source code to understand how linux works. (Although, it may work if you are sufficiently genius.)
So, first try to get a doc explaining what the piece of code does. Having an insight into the working certainly makes browsing the code a lot simpler and faster. Then, use whatever-piece-of-software you have so that you know the inputs and outputs. Start with the source code only when you think you have a good understanding of the product.

Don't worry if you cannot seem to understand or remember anything the first time. Be persistent and you'll get the knack of it.

Browsing the source, btw, is a very good way to understand how people think. So, if you have a crush, go get her source code! :P

grep-ing your way to glory!

Grep is oxygen. Also, Grep is a very very powerful content search tool found on Unix-based systems.

Basic Syntax:
>> grep [options] [pattern] [files to be searched]
eg. >> grep -r Hello .
(Grep the current directory for the string "Hello". Make the search recursive i.e. search within sub-directories too)

grep searches in all files. Even binary files. This can sometimes be irritating, since results from binary files are seldom useful. Pass a "-I" ignore binary files. (grep examines the first few of the file to determine whether the file is binary or ASCII. extensions are NOT used)

Also, you may want to ignore case. Use "-i"

By default, the format of printing the output is
"[path-to-file]: [line containing the matching pattern]"
Sometimes it might be useful to alter this pattern.
Use
1) "-n" to display the line number in the file where a match was found
2) "-o" to show only the matching text and NOT the entire line.
3) "-C [num of lines]" to show the context (before and after)
4) "--color" to show the matching pattern in color.


Also, sometimes you might be interested in only the number of matching lines. use "-c".

Another VERY useful option is "invert-match" (i.e. select non-matching lines)
use "-v".
eg. >> grep -v -e "^#" my.c
(i.e. select all lines that do NOT start with a #)

grep can take only basic regular expressions. Use "-e" to pass an expression which uses the extended regex format.

These are pretty much the frequently used options. man grep for more options.

Some Grep usage scenarios:
grep alongwith piping makes a great combo!
you can say something like
>> find . -name "*.c" | xargs grep -in Hello
to search only C files within a directory.
Or you can say
>> find . -name "*.c" | grep hello | grep -v not_welcome
to find a filename containing the string "hello" in it but NOT "not_welcome" !!

NOTES:
1) I purposely havn't talked about regular expressions. They are worthwhile to learn since they are required everywhere, not only for grep. With a good knowledge of regular expressions, grep is awesomely powerful!
2) grep is slow. Understandable. since it is content search!
3) Are more examples required? Let me know!

PS:
So, grep is used for search. But, what about search and replace?
Well, use "sed" for that. sed is good! (sed has saved me loads and loads of time. It is good to know atleast basic sed)
A nice tutorial can be found here.

EOF

So what is so great about find?

Why do people not like find? It is a nice command to have. I gather that there is no "find" in BSD and that find is "impure" (I am not sure about both) Well, frankly, it makes no difference to me, as a user, whether a command is pure or impure. Find is a simple slick and helpful command and that is what I care about. As for BSD, get the sources and compile!

LOCATE:
Anyway, there is an alternative to find (albeit lame). It is "locate".
Basic syntax :

>> locate [pattern]

(Don't mind the square brackets. '<' brackets makes google think that I am writing some xml doc. So, a quick fix) Locate merely searches the database of names that was created by "updatedb" command. updatedb is periodically scheduled by crond. So, your database is automatically updated. You don't need to worry about that. One advantage of this approach is that the search is lightning fast. If you compare the performance of locate and find, locate would win by light years! Also, the syntax of locate is deceptively simple. Merely, "locate {pattern}"
But, as every girl you like is always taken, locate has its own drawbacks.

1) locate command doesn't check whether the file actually exists on your machine. It simply consults the database and gives back the results. (Thus, after an updatedb, you delete a file, you still get it in the locate results until crond schedules another updatedb). You can pass "-e" to locate to avoid this condition. -e checks whether the file actually exists.

2) If you create a new file, it won't appear in the locate results for a while. (Until crond schedules an updatedb)

3) There is no way you can ask locate to search only within a particular directory. You have to search the entire directory tree, always. This, in my opinion, is the worst drawback of locate.

Some other options that could be passed to locate:
-i : ignore case
-b : match basename only. (i.e. the final component of the whole name)
-e : check if file exists
-E : print only files that do not exist, but were there when database was made.


FIND:
Find is good.
With find you can search a part of the direcotry tree. The basic syntax is

>> find [directory to search in] [options]

eg. >> find . -name "Linux"
the beauty of the command is that, you can have many many search parameters. Like:
1) -name [pattern]
Search name of the file
2) -user [username]
Search for files owned by this particular user
3) -perm -{octal permission bits}
Search for files with this permissions.
4) -type [c]
Search for files only of type . (eg. b for block. p for pipe etc etc)
and many more. Check out the man page. You can combine the above options.
eg. >> find . -name "Linux" -user httpd

Also, with find the expression part can be "expression" "!expr" "expr or expr" "expr and expr" etc etc. Quite handy!

What more? You can format your output as per you wish. check the -print option in the find man page. Havn't found the need to use this one, but such options are good for writing scripts.

When you search in "/", find unecessarily searches the mounted windows paritions too? No prob, pass "-mount" and it won't !! It'll skip all mounts.

With find, you can have more control over the search. But the downside is that find is terribly and horribly slow.


As a third lame option. You can define an alias. Something like

>> alias myfind='updatedb; locate'

So, "myfind" becomes a new command which expands to 'updatedb; locate'. Neat, eh?
although, the overhead of updatedb might not always be acceptable. In those cases, you can simply use locate. updatedb, btw, is NOT very slow. So, it is a better option than find, if you don't want too tight a control on your search.

Enjoy searching!!

(PS: Lookout for the next post on grep)

Emacs v/s vi !

While looking for a comparison among the editors I stumbled across these. Great!!
(Both emacs and vi are great and masochistic. Both of them have a steep learning curve but equally great are their uses. You can find a good comparison here.)

Emacs and vi have been involved in a never-ending holy war since ages. All of the verbal battles involve critisizing the people using the editor. I love such wars! :D. Yes. What with all the insulting and swearing :D. Here is a sample....

LOL ----->>

How to ask questions?

n00bs often get frustrated when they start using Linux. Partly because it is very different than Windows and partly because they lack patience. Patience to figure out what is not working and why it is not working. Patience to learn. They want quick answers and so they turn to forums and mailing lists. We have seen hundreds of stupid posts/mails which go something like this - "I can't get xyz working. Help me. NOW!".

One thing we all should understand is the people who made the software are not your servants. You should be thankful they opted to share their effort for free. Isn't it your responsibility to atleast try somethings out before wasting their precious time?? You get to learn a lot in the process too. Its a win-win situation!!

So, before posting any question to a mailing list/forum please make sure thatyou have done your best but couldn't find the solution. (Google. Ask around. Have a look at the debugging output) Let others know what you have tried. Write it in your mail/post. Be respectful and humble.

There is a nice document which documents this particular aspect. You can find it here.

Happy getting-things-to-work!!! :D

Nothing technical about it !!

So, we are supposed to mail our status reports each week to our manager. One of the guys (some people have a good good idea who he is !!) wrote this in his status report:

Tasks this week:
* Worked on "one build to rule them all" infrastructure


And bang came the reply from the manager:

Three Builds for the Linux-kings in the MSI,
Seven for the SQA-lords in their shield-halls of stone,
Nine for MIPL Mortal Men doomed to code,
One for the Dark Lord Kedar on his Bamboo throne

In the Land of Linux where the RPMs lie.
One Build to rule them all, One Build to find them,
One Build to bring them all and in the flash-ness bind them
In the Land of Linux where the RPMs lie.


Need I say more?

Coding Style...

No technical blog can start without commenting on the coding style. 90% of the times the code written by you would have to be understood by a third person who will enhance or maintain it. (The other 10% being college assignments.. even there the code has to be understood by MANY third people :P). So, wouldn't it be nice if your code was easily understandable and readable?

You will understand the importance of coding style once you start studying code written by others. Some get so frustrated that they re-indent the entire code base (:P) which leads to worse problems!! :P. So, better write understandable code. Coding is an art and an art is nothing without presentation!

A very good guide to coding style can be found in the linux kernel documentation.
Just check out Documentation/CodingStyle in the Linux kernel sources. Remember that this coding style is recommended by people who have written/maintained millions of LOC. It has to be the best. And it is!

The file also provides a configuration to be integrated with emacs so that emacs does some of the things automatically for you. Emacs is great!

Welcome to the world where we like to speak quality C !! :)

The traditional post....

As is the tradition, the first post of any blog always explains what the blog is going to be about. And as is the tradition, the blog is never about it !!

We'll stick to the tradition.
So here goes.
The blog will contain posts that I don't think are fit for my other blogs. My experiences and Humour. Basically everything that people don't think is fun. I won't say that I agree with them, but I can understand. Some things are better left to geniuses :P:D.
Anyway, expect technical or atleast no non-technical posts from this blog!

And have fun!
Amen!