Category: File system

Creating virtual disks using dd and losetup.

To create an image file, in this case a “virtual disk”, use “dd” command. The below command will write zeros to a file of a specified size.

dd if=/dev/zero of=1GB_disk.img bs=1M count=1024

Once completed, a partition can be created using cfdisk or fdisk command. Then the filesystem should be created using mkfs.ext4

cfdisk 1GB_disk.img

Now, you can proceed to setup a loop device for you image. This requires the use of “losetup”. This command will assign an available loop device (-f option to find one) to the partition on the image, and show the name a loop device (–show option):​

losetup -Pf --show 1GB_disk.img

If successful, you should be able to access the partition by mounting the image.

[root@s1 disk]# lsblk|grep loop
loop0 7:0 0 1G 0 loop /mnt/disk
[root@s1 disk]#

mount /dev/loop0 /mnt/disk

[root@s1 disk]# df -hP /mnt/disk/
Filesystem Size Used Avail Use% Mounted on
/dev/loop0 976M 46M 863M 6% /mnt/disk
[root@s1 disk]#

To remove a loop device just run:

losetup -d /dev/loop0

Removing volume group and logical volume after physical drive has been removed

root:/ # lvs
/dev/5gbdisk_vg/5gbdisk: read failed after 0 of 4096 at 1073676288: Input/output error
/dev/5gbdisk_vg/5gbdisk: read failed after 0 of 4096 at 1073733632: Input/output error
/dev/5gbdisk_vg/5gbdisk: read failed after 0 of 4096 at 0: Input/output error
/dev/5gbdisk_vg/5gbdisk: read failed after 0 of 4096 at 4096: Input/output error
/dev/sdc: read failed after 0 of 4096 at 0: Input/output error
/dev/sdc: read failed after 0 of 4096 at 10737352704: Input/output error
/dev/sdc: read failed after 0 of 4096 at 10737410048: Input/output error
/dev/sdc: read failed after 0 of 4096 at 4096: Input/output error
LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert
home sp3tosp4 -wi-ao--- 4.00g
var sp3tosp4 -wi-ao--- 8.00g
root:/ #

When the disk was physically removed, the /dev/sdc and this device nodes wasn’t automatically removed. The above errors are clearly indicating that /dev/sdc and /dev/myvg/mylv can no longer be read due to the removal of the disk.
Remove the stale /dev/sdc device node and clean up the stale device-mapper nodes. In the above example, this would be accomplished by either a simple reboot, or by running the following:

root:/ # dmsetup remove –force /dev/5gbdisk_vg/5gbdisk
root:/ # echo 1 > /sys/block/sdc/device/delete

root:/ # pvs
PV VG Fmt Attr PSize PFree
/dev/sdb sp3tosp4 lvm2 a-- 16.00g 4.00g
root:/ # lvs
LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert
home sp3tosp4 -wi-ao--- 4.00g
var sp3tosp4 -wi-ao--- 8.00g
root:/ #

When listing a directory on an NFS share, directory listing fails half way through with ‘ls: reading directory .: Too many levels of symbolic links

Disable the dir_index filesystem feature on the filesystem that is being exported from the NFS server.

root# tune2fs -O ^dir_index /dev/sdXY, where sdXY is your device.

To re-enable the dir_index feature use the command:

root# tune2fs -O dir_index /dev/sdXY, where sdXY is your device.

comm – compare two sorted files line by line.

Usage: comm [OPTION]… FILE1 FILE2
Compare sorted files FILE1 and FILE2 line by line.

With no options, produce three-column output. Column one contains
lines unique to FILE1, column two contains lines unique to FILE2,
and column three contains lines common to both files.

-1 suppress column 1 (lines unique to FILE1)
-2 suppress column 2 (lines unique to FILE2)
-3 suppress column 3 (lines that appear in both files)

–check-order check that the input is correctly sorted, even
if all input lines are pairable
–nocheck-order do not check that the input is correctly sorted
–output-delimiter=STR separate columns with STR
–help display this help and exit
–version output version information and exit

Note, comparisons honor the rules specified by ‘LC_COLLATE’.

Examples:
comm -12 file1 file2 Print only lines present in both file1 and file2.
comm -3 file1 file2 Print lines in file1 not in file2, and vice versa.

GNU coreutils online help:
Full documentation at:
or available locally via: info ‘(coreutils) comm invocation’

File attributes – chattr and lsattr.

File attributes

chattr – (Change Attribute) is a command line Linux utility that is used to set/unset certain attributes to a file in Linux system to secure accidental deletion or modification of important files and folders, even though you are logged in as a root user.
lsattr – list file attributes on a Linux second extended file system
By default, file attributes are not preserved by cp, rsync, and other similar programs.

For ext2 and ext3 file systems, the e2fsprogs package contains the programs lsattr and chattr that list and change a file’s attributes, respectively. Though some are not honored by all file systems, the available attributes are:

a: append only
c: compressed
d: no dump
e: extent format
i: immutable
j: data journalling
s: secure deletion
t: no tail-merging
u: undeletable
A: no atime updates
C: no copy on write
D: synchronous directory updates
S: synchronous updates
T: top of directory hierarchy

For example, a file is set with “i” attribute, cannot be modified (immutable). Means no renaming, no symbolic link creation, no execution, no writable, only superuser can unset the attribute.

root# chattr +i /path/to/file

A file is set with “a” attribute, can only be open in append mode for writing.

root# chatt +a /path/to/file

To remove an attribute on a file just change + to -.

To list the file atributes, use the lsattr command:

root# lsattr /path/to/file
----ia---------- /path/to/file
root#

Mount an ISO image

An ISO image or .iso (International Organization for Standardization) file is an archive file that contains a disk image called ISO 9660 file system format. Every ISO file have .ISO extension has defined format name taken from the ISO 9660 file system and specially used with CD/DVD rom’s. In simple words an iso file is a disk image.
ISO images can be created from optical discs by disk imaging software, or from a collection of files by optical disc authoring software, or from a different disk image file by means of conversion. Software distributed on bootable discs is often available for download in ISO image format. And like any other ISO image, it may be written to an optical disc such as CD or DVD.

Mount an ISO image:
root# mount -t iso9660 -o loop /root/debian-8.2.0-amd64-xfce-CD-1.iso /mnt/iso/

Unmount an ISO image:
root# umount /mnt/iso

/ on LVM gets mounted by initrd with kernel device name /dev/dm-X instead of /dev/mapper/XXX name

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=791754

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=791754;msg=10;filename=resolve_device.diff;att=1

— /usr/share/initramfs-tools/scripts/functions.orig 2015-07-08 10:48:26.000000000 +0200
+++ /usr/share/initramfs-tools/scripts/functions 2015-07-08 10:39:15.000000000 +0200
@@ -324,6 +324,10 @@
UUID=*)
DEV=”/dev/disk/by-uuid/${DEV#UUID=}”
;;
+ /dev/mapper/*)
+ echo “$DEV”
+ return 0
+ ;;
esac
# Only canonicalise if a valid file, in case $DEV isn’t a filename
[ -e “$DEV” ] && DEV=$(readlink -f “$DEV”)

And then run the following command:

update-initramfs -u -k all

If this is not possible to implemented this patch/workaround always this command can be used:

dmsetup ls --tree

ACL and a sticky bit.

Set a directory with rwx inherit default permissions for groups, also to ensure that all files and directories crated in will have ownership the “users” group.

All new files created in the directory will have the group set to the group of the directory.

chmod g+s user1

Set the set file access control lists (ACL), set the group to rwx default and others to read access only.

setfacl -d -m g::rwx user1
setfacl -d -m o::r user1

Output:

server:/home # l user1/
total 393256
drwxr-xr-x 12 root root 4096 Jul 24 10:38 ./
drwxr-xr-x 26 root root 4096 Oct 16 15:03 ../
drwxrwsr-x+ 4 user1 users 4096 Oct 28 15:16 user1/
drwxr-xr-x 2 user2 users 4096 Oct 28 15:15 user2/
drwxr-xr-x 6 user3 dba 4096 Oct 13 14:24 user3/
drwx------ 2 root root 16384 Apr 17 2014 lost+found/

server:/home> getfacl user1/
# file: user1/
# owner: user1
# group: users
# flags: -s-
user::rwx
group::rwx
other::r-x
default:user::rwx
default:group::rwx
default:other::r--

server:/home/user1> l
total 72
drwxrwsr-x+ 4 user1 users 4096 Oct 28 15:36 ./
drwxr-xr-x 12 root root 4096 Jul 24 10:38 ../
-rw------- 1 user1 users 2197 Oct 28 15:14 .bash_history
drwxrw-r-- 2 user1 users 4096 Oct 27 08:43 bin/
-rw-rw-r-- 1 user2 users 0 Oct 28 15:36 createdbyuser2.txt
-rw-rw-r-- 1 user1 users 10324 Oct 21 09:22 createdbyuser1.txt
server:/home/user1>

tmpfs

tmpfs is a temporary filesystem that resides in memory and/or your swap partition(s), depending on how much you fill it up. Mounting directories as tmpfs can be an effective way of speeding up accesses to their files, or to ensure that their contents are automatically cleared upon reboot.

By default, a tmpfs partition has its maximum size set to half your total RAM, but this can be customized. Note that the actual memory/swap consumption depends on how much you fill it up, as tmpfs partitions do not consume any memory until it is actually needed.
To explicitly set a maximum size, in this example to override the default /tmp mount, use the size mount option:

/etc/fstab
tmpfs /tmp tmpfs nodev,nosuid,size=4G 0 0

The tmpfs can also be temporarily resized without the need to reboot:

root# mount -o remount,size=6G,noatime /tmp

To find a filesystem at /proc/mounts of /tmp use findmnt which will list all mounted filesytems or search for a filesystem. The findmnt command is able to search in /etc/fstab, /etc/fstab.d, /etc/mtab or /proc/self/mountinfo. If device or mountpoint is not given, all filesystems are shown.

root# findmnt --target /tmp
TARGET SOURCE FSTYPE OPTIONS
/tmp tmpfs tmpfs rw,nosuid,nodev,relatime