In a Linux filesystem, a file is actually a link to an inode, which contains the file’s properties, such as ownership, permissions, addresses of the data blocks .etc. When you delete a file using the rm command, you’re removing the link that points to its inode, but not the inode; other processes can still have it open. After they’re done, and all links are removed that an inode and the data blocks it pointed to it are made available for writing.

Now, if a process still has the file open, the data’s there somewhere, even though according to the directory listing the file already appears to be gone. Here is where the Linux process pseudo-filesystem, the /proc directory, comes into action. Every process on the system has a directory with its name on it, inside of which lies lots of things for eg: file descriptor which is a subdirectory containing links to all files that the process has opened. Even if a file has been removed from the filesystem, a copy of the data will be right there:

/proc/process id/fd/file_descriptor

Now to find it, you need to get the id of the process which has the file open, and the file descriptor. These you get with lsof, which means “list open files.”

Once you get that information from lsof, you can copy the data out of /proc and get what you had lost (virtually).

You can test this with the following example

First, create a text file that you can delete and then bring back:

$ man lsof | col -b > test_file

Then have a look at the contents of the file that you just created:

$ less test_file

You should see a plaintext version of lsof’s huge man page looking out at you, courtesy of less.

Now press Ctrl-Z to suspend less. Back at a shell prompt make sure your file is still there:

$ ls -l test_file

-rw-r–r– 1 arun arun 117478 Feb 22 05:51 test_file

[arun@server /]# stat test_file

File: `test_file’

Size: 117478 Blocks: 240 IO Block: 4096 regular file

Device: 41h/65d Inode: 50201393 Links: 1

Access: (0644/-rw-r–r–) Uid: ( 0/ arun) Gid: ( 0/ arun)

Access: 2011-02-22 05:52:04.000000000 -0800

Modify: 2011-02-22 05:51:32.000000000 -0800

Change: 2011-02-22 05:51:32.000000000 -0800

$ rm test_file

$ stat test_file

stat: cannot stat `test_file’: No such file or directory

Here, we must not allow the process still using the file to exit, because once that happens, the file will really be gone and your troubles will intensify. Now, if this were a video or sound file that you were playing, the first thing to do at the point where you realize that you have deleted the file would be to immediately pause the application playback, or otherwise freeze the process, so that it doesn’t eventually stop playing the file and exit.

Now to bring the file back. First, see what lsof has to say about it:

$lsof | grep test_file

less 14094 arun4r REG 0,65 117478 50201393 (deleted) /test_file

The first column gives you the name of the command associated with the process, the second column is the process id, and the number in the fourth column is the file descriptor (the “r” means that it’s a regular file). Now you know that process 14094 still has the file open, and you know the file descriptor, 4. That’s everything you have to know to copy it out of /proc.

We might think that using the -a flag with cp is the right thing to do here since we are restoring the file, but it’s actually important that you don’t do that. Otherwise, instead of copying the literal data contained in the file, you’ll be copying only a now-broken symbolic link to the file as it once was listed in its original directory:

$ ls -l /proc/14094/fd/4 test_file

lr-x—— 1 arun arun 64 Feb 22 05:53 /proc/14094/fd/4 -> (deleted) /test_file

$ cp /proc/14094/fd/4 test_file

And finally, verify that you’ve done well:

$ ls -l test_file

-rw-r–r– 1 arun arun 117478 Feb 22 05:55 test_file

$ man lsof | col -b > test_file1

$ cmp test_file test_file1

No difference in files, which shows that the restoration was successful.

If you require help, contact SupportPRO Server Admin