Monday, March 13, 2017

Introduction to Virtualization on Solaris and Linux

Virtualization describes a technology in which an application, guest OS or data storages is abstracted away from the true underlying hardware or software.
Key use of virtualization technology is server virtualization.
The performance of the virtual system is not equal to the performance of the OS running on the true hardware; the concept of virtualization works because most guest OS and application don’t need the full use of the underlying hardware.
This allows for greater flexibility, control and isolation by removing the depending on n a given hardware platform.
Whilst initially mean for server virtualization, the concept of virtualization has spread to applications, networks, data and desktops.


KVM:
Kernel-based Virtual Machine (KVM) is a free, open source virtualization architecture for Linux distributions.
KVM virtualization, which is supported by RedHat; uses Type-2 hypervisor that resides within the Linux kernel.  KVM virtualization is often compared with Xen.


Hypervisor:
A hypervisor is a function which abstracts-isolates -OS and applications from the underlying computer hardware.  This abstraction allows the underlying host machine hardware to independently operate one or more virtual machines as guests, allowing multiple guest VMs to effectively share the system’s physical compute resources, such as processor cycles, memory space, network bandwidth and so on.

A hypervisor is sometimes also called a virtual Machine Monitor (VMM).

A hypervisor makes the underlying hardware details irrelevant to the VMs.  This allows any VMs to be moved or migrated between any local or remote virtualized servers – with sufficient computing resources available – almost at- will with effectively zero disruption to the VM; a feature often termed as “Live Migration”.
VMs are also logically isolated from each other, even though they run on the same physical machine.
In effect, a VM has no native knowledge or dependence on any other VMs.  An error, crash or malware attack on one VM does not proliferate to other VMs on the same or other machines.  This makes hypervisor technology extremely secure.

Types:
Type-1: Bare Metal
Type-2: Hosted

Type-1/Bare Metal:
Hypervisors are deployed directly a top the system’s hardware without any underlying OS or other software.  These are called “Bare Metal” hypervisors and are the most common and popular type of hypervisor for the enterprise data center.
Eg: VSphere or Hyper-V


Type-2/Hosted:

Hypervisors runs as a software layer atop a host OS and are usually called “Hosted” hypervisors like VMware. Hosted hypervisors are often found on end points like PCs.
The role of hypervisor is expanding.

Storage Hypervisors are used to virtualize all of the storage resources in the environment to create centralized storage pools that administrators can provision, without having to concern themselves with where the storage was physically located.

Networks are also being virtualized with hypervisors, allowing networks and network devices to be created, changes, managed and destroyed entirely through software without ever touching physical network devices.


Solaris Zones:
Zones is a software partitioning technology that enables the creation and management of multiple virtualized OS execution environments within a single instance of the Solaris Kernel.  Each zone (virtualized environment) appears as a system to the processes, users, and administrators within the zone and is isolated from other zones running within the same kernel instance.
Note: theoretically/ conceptually – The upper limit for the number of zones on a system is 8192.

The isolation provides security, since processes running in one zone are not visible to the processes running in other zones in the same kernel instance.  The only exception to this is the global zone, which is the primary zone that represents the Solaris kernel instance.  All processes running in all zones in a kernel instance are visible to the global zone.
Zones also provide a resource management container, such that zones created to run specify applications (webserver, database server etc) can be configured to use a subset of the hardware resources available on the system.

Caution:
Some refer to Zones and Containers interchangeably as if they mean exactly the same thing.  This is incorrect because containers is a technology that comprises the resource management features, such as resource pools and solaris zones.  Solaris zones is a subset of containers, so the two terms should not be used interchangeably.


Containers:
1.       Containers = Zones + SRM (Solaris Resource Management)
2.       Containers is a technology that comprises the resource management features, such as resource pools and solaris zones
3.       Remember solaris containers use the same kernel structure.



Resource Management:
1.       One of the integral component of Soalris10 containers technology.
2.       Allows us to perform –
a.       Allocate specific computer resource, such as CPU times & memory
b.       Monitor how resource allocations are being used and adjust the allocation when required.
c.       Generates more detailed accounting information
d.       A new resource capping daemon (rcapd) by a project. [Remember, a project can be number of processes/users].


Consolidation:
1.       Resource Management feature of Solaris containers is extremely useful when a need of consolidating a number of applications to run on a single server.
2.       Reduces cost, complexity of having to manage numerous separate systems.
3.       Consolidate applications onto fewer, larger, more scalable servers, and also segregate the work load to restrict the resources that each can use.
4.       Via Resource Management feature, multiple workloads can now be run on a single server, providing an isolated environment for each, so that one workload cannot affect the performance of the other.
5.       Resource pools can be utilized to group applications, functions, together and control their resource usage globally, such as the maximum amount of CPU resource or memory.  Additionally, the resource management feature can tailor the behavior of the Fair Share Scheduler (FSS) to give priority to specific applications.  This is very useful if, to allocation additional resources to a group of resources for a limited period of time.
Before Resource Management, was introduced, this would have meant that a larger server would be needed to accommodate the resource requirement; even though it only  would be used to its capacity once in a month.  Now the resources can be allocated accordingly to priority, allowing the server to be more effectively utilized.



LDOM:
1.       In virtualization terminology, Logical DOMains (LDOM) is a technology for server virtualization that enable users to allocate system resources, such as memory or devices into logical groupings in order to create multiple virtual machines.
2.       Each of the these virtual machines can use its own OS, have its own resources and also be identified as a single stand-alone unit within the computer system.
3.       LDOM is a free Sun virtualization technology and is supported on Sun servers which utilize Ultra SPARC T1 or Ultra SPARC T2 processors, running at least Solaris 10 11/06 and the latest server firmware.
4.       LDOMs make use of a hypervisor as a layer of abstraction between real, physical hardware and virtual hardware.  This virtual hardware is then used to create number of guest system which can behave very similar to a system running on bare metal.
5.       Importantly, each has its own OBP, each will install its own copy of the Solaris OS and each will see a certain amount of CPU, memory, disk and network resources available to it.
6.       Unlike some other Type1 hypervisors running on X86 hardware, the SPARC hypervisor is embedded in the system firmware and makes both of supporting functions in the Sun4V SPARC instruction set as well as the overall CPU architecture to fulfill its function






Sunday, November 27, 2016

Red Hat Linux: YUM Package Administration


2.       Yellowdog Updater Modified.
3.       Auto dependency resolution.
4.       Ability to specify multiple package names.
5.       Only one repostiry will be active. So kindly ensure that respositry is updated.
6.       Repositry can be available either –
a.       Locally
b.       FTP server
c.       HTTP server

  
Createrepo:
1.       Creates sub-directories (repodata) ancillary files and various DB files to serve the packages to ‘yum’ clients.
2.       Ensure that ‘createrepo’ package is installed before running/creating local repo directory; running the command ‘createrepo’.
3.       ‘createrepo’ command queries all current packages (differ to each version) and generates a SQLite DB and anciallary files beneath: ‘repodata’ directory.
4.       Ensure the creation of a valid repo file for the locally.


Output:
 [root@TestServer yum.repos.d]# createrepo /etc/yum.repos.d/packages
(Note: create local repostiry using “createrepo” command for building the local yum reposity)



 Where /etc/yum.repos.d/packages is the path of the location where the packages are stored locally to the system.

Output:

[root@TestServer yum.repos.d]# yum list repo
Loaded plugins: product-id, refresh-packagekit, security, subscription-manager
Updating certificate-based repositories.
Unable to read consumer identity
packages                                                                                           | 1.3 kB     00:00 ...
packages/primary                                                                                   | 1.7 MB     00:00 ...
packages                                                                                                        3653/3653
Error: No matching Packages to list


 /etc/yum.conf:
1.       Configuration file.
2.       Specifies the yum log file.
Default log file: /var/log/yum.log
Provides the information about recently installed packages to the server/yum client.
3.       Defines where the repo directory has to be placed/located/created.

‘repos’ file:
[abcd]   header file.
name=ABCD   à   name of the repositry
                baseurl=  base url path, where the packages and repo directory is created.


# yum info  
1.       Provides the status of the packages installed.
2.       Provides the details about the specified package description, type, release, arch of the packages.
3.       Lists package meta data.


# yum search
Search whether the package is installed, available on the repository.


# yum history          Much used as the time of auditing.
1.       Lists out the usage/transaction
2.       What has been carried out, i.e, install/remove/re-install/upgrade
3.       Who performed the action.


# yum repolist         Lists out the created/available repository created.


# yum install
1.       To install packages once.
2.       Can also install multiple packages at a time (# yum install ).

Output:
[root@TestServer yum.repos.d]# yum install http*
Loaded plugins: product-id, refresh-packagekit, security, subscription-manager
Updating certificate-based repositories.
Unable to read consumer identity
Setting up Install Process
Package httpd-2.2.15-15.el6_2.1.x86_64 already installed and latest version
Package httpd-tools-2.2.15-15.el6_2.1.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package httpd-devel.x86_64 0:2.2.15-15.el6_2.1 will be installed
--> Processing Dependency: apr-util-devel for package: httpd-devel-2.2.15-15.el6_2.1.x86_64
--> Processing Dependency: apr-devel for package: httpd-devel-2.2.15-15.el6_2.1.x86_64
---> Package httpd-manual.noarch 0:2.2.15-15.el6_2.1 will be installed
--> Running transaction check
---> Package apr-devel.x86_64 0:1.3.9-3.el6_1.2 will be installed
---> Package apr-util-devel.x86_64 0:1.3.9-3.el6_0.1 will be installed
--> Processing Dependency: openldap-devel for package: apr-util-devel-1.3.9-3.el6_0.1.x86_64
--> Processing Dependency: expat-devel for package: apr-util-devel-1.3.9-3.el6_0.1.x86_64
--> Processing Dependency: db4-devel for package: apr-util-devel-1.3.9-3.el6_0.1.x86_64
--> Running transaction check
---> Package db4-devel.x86_64 0:4.7.25-17.el6 will be installed
--> Processing Dependency: db4-cxx = 4.7.25-17.el6 for package: db4-devel-4.7.25-17.el6.x86_64
--> Processing Dependency: libdb_cxx-4.7.so()(64bit) for package: db4-devel-4.7.25-17.el6.x86_64
---> Package expat-devel.x86_64 0:2.0.1-11.el6_2 will be installed
---> Package openldap-devel.x86_64 0:2.4.23-26.el6 will be installed
--> Processing Dependency: cyrus-sasl-devel >= 2.1 for package: openldap-devel-2.4.23-26.el6.x86_64
--> Running transaction check
---> Package cyrus-sasl-devel.x86_64 0:2.1.23-13.el6 will be installed
---> Package db4-cxx.x86_64 0:4.7.25-17.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

===========================================================================Package                          Arch                   Version                           Repository                Size
===========================================================================Installing:
 httpd-devel                      x86_64                 2.2.15-15.el6_2.1                 packages                 147 k
 httpd-manual                     noarch                 2.2.15-15.el6_2.1                 packages                 781 k
Installing for dependencies:
 apr-devel                        x86_64                 1.3.9-3.el6_1.2                   packages                 176 k
 apr-util-devel                   x86_64                 1.3.9-3.el6_0.1                   packages                  69 k
 cyrus-sasl-devel                 x86_64                 2.1.23-13.el6                     packages                 302 k
 db4-cxx                          x86_64                 4.7.25-17.el6                     packages                 588 k
 db4-devel                        x86_64                 4.7.25-17.el6                     packages                 6.6 M
 expat-devel                      x86_64                 2.0.1-11.el6_2                    packages                 120 k
 openldap-devel                   x86_64                 2.4.23-26.el6                     packages                 1.1 M

Transaction Summary
===========================================================================
Install       9 Package(s)

Total download size: 9.8 M
Installed size: 37 M
Is this ok [y/N]:


Downloading Packages:
---------------------------------------------------------------------------
Total                                                                                      64 MB/s | 9.8 MB     00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
  Installing : apr-devel-1.3.9-3.el6_1.2.x86_64                                                                       1/9
  Installing : expat-devel-2.0.1-11.el6_2.x86_64                                                                      2/9
  Installing : db4-cxx-4.7.25-17.el6.x86_64                                                                           3/9
  Installing : db4-devel-4.7.25-17.el6.x86_64                                                                         4/9
  Installing : cyrus-sasl-devel-2.1.23-13.el6.x86_64                                                                  5/9
  Installing : openldap-devel-2.4.23-26.el6.x86_64                                                                    6/9
  Installing : apr-util-devel-1.3.9-3.el6_0.1.x86_64                                                                  7/9
  Installing : httpd-devel-2.2.15-15.el6_2.1.x86_64                                                                   8/9
  Installing : httpd-manual-2.2.15-15.el6_2.1.noarch                                                                  9/9
Installed products updated.
  Verifying  : openldap-devel-2.4.23-26.el6.x86_64                                                                    1/9
  Verifying  : cyrus-sasl-devel-2.1.23-13.el6.x86_64                                                                  2/9
  Verifying  : apr-util-devel-1.3.9-3.el6_0.1.x86_64                                                                  3/9
  Verifying  : db4-cxx-4.7.25-17.el6.x86_64                                                                           4/9
  Verifying  : expat-devel-2.0.1-11.el6_2.x86_64                                                                      5/9
  Verifying  : db4-devel-4.7.25-17.el6.x86_64                                                                         6/9
  Verifying  : apr-devel-1.3.9-3.el6_1.2.x86_64                                                                       7/9
  Verifying  : httpd-manual-2.2.15-15.el6_2.1.noarch                                                                  8/9
  Verifying  : httpd-devel-2.2.15-15.el6_2.1.x86_64                                                                   9/9

Installed:
  httpd-devel.x86_64 0:2.2.15-15.el6_2.1                      httpd-manual.noarch 0:2.2.15-15.el6_2.1

Dependency Installed:
  apr-devel.x86_64 0:1.3.9-3.el6_1.2     apr-util-devel.x86_64 0:1.3.9-3.el6_0.1  cyrus-sasl-devel.x86_64 0:2.1.23-13.el6
  db4-cxx.x86_64 0:4.7.25-17.el6         db4-devel.x86_64 0:4.7.25-17.el6         expat-devel.x86_64 0:2.0.1-11.el6_2
  openldap-devel.x86_64 0:2.4.23-26.el6
Complete!




NOTE:

If the package is already installed/present/available version, and the installing version is same, then no action will be taken.




# yum reinstall
1.       To reinstall the package, when the package is already installed.
2.       The above command functions similar to # rpm –replace option.
3.       Imagine if the package is NOT installed it will NOT work. Means,  it will NOT install the package.


# yum –u reinstall
Where –
-y = yes  à to predefine yes, so that the package will be installed.


# yum erase  à  to erase/remove/delete the installed packages.


# yum –y erase
Where –
-y = yes à to predefine yes.
Will NOT check with the yum server/repository while removing/deleting the package.
It checks the local system, if the package is installed, it removes/deletes it.

# yum deplist
Displays the dependencies and their providers.

# yum localinstall
1.       To install packages from the local file system.
2.       While locally installing the package, complete name of the package along with the version has to be specified.