Tuesday, November 10, 2009

Capacity Planning

Capacity planning is the process of determining the production capacity needed by an organization to meet changing demands for its products.[1] In the context of capacity planning, "capacity" is the maximum amount of work that an organization is capable of completing in a given period of time.

A discrepancy between the capacity of an organization and the demands of its customers results in inefficiency, either in under-utilized resources or unfulfilled customers. The goal of capacity planning is to minimize this discrepancy. Demand for an organization's capacity varies based on changes in production output, such as increasing or decreasing the production quantity of an existing product, or producing new products. Better utilization of existing capacity can be accomplished through improvements in overall equipment effectiveness (OEE). Capacity can be increased through introducing new techniques, equipment and materials, increasing the number of workers or machines, increasing the number of shifts, or acquiring additional production facilities.

Capacity is calculated: (number of machines or workers) × (number of shifts) × (utilization) × (efficiency).

The broad classes of capacity planning are lead strategy, lag strategy, and match strategy.

* Lead strategy is adding capacity in anticipation of an increase in demand. Lead strategy is an aggressive strategy with the goal of luring customers away from the company's competitors. The possible disadvantage to this strategy is that it often results in excess inventory, which is costly and often wasteful.
* Lag strategy refers to adding capacity only after the organization is running at full capacity or beyond due to increase in demand (North Carolina State University, 2006). This is a more conservative strategy. It decreases the risk of waste, but it may result in the loss of possible customers.
* Match strategy is adding capacity in small amounts in response to changing demand in the market. This is a more moderate strategy.

In the context of systems engineering, capacity planning is used during system design and system performance monitoring.

Capacity planning is long-term decision that establishes a firm of overl level of resources. It extends over time horizon long enough to obtain resources. Capacity decision affects the production lead time, customer responsiveness, operating cost and company ability to compete. Inadequate capacity planning can lose the customer and business. Excess capacity can drain the company's resources and prefect investment of lucrative venture. When capacity should be increased and how much increase are the critical decisions.

Click Here for Capacity Planning White Paper.

Monday, November 9, 2009

Websphere Automation Tool ( WASIC)

WASIC is Websphere Application Server Installation and Configuraton Automation Tool.

Overview:-

WASIC is a tool to install, configure and administrate Websphere application server Version 6.1/7.0.

Features of WASIC:

1. Deployment Manager Installation.
2. Node Installation
3. Federate node to Deployment Manager.
4. Creating Profiles.
5. Install Update installers
6. Apply Fix Packs on Base Installation.
7. Changes port of deployment Manager
8. Checks whether ports are already in use.
9. Changes and checks node agent ports.
10.Webserver Installation and configure them with Application Server.
11.Cluster and server creation and configuring servers.
12.Create Cluster.
13.Create server and add the server to cluster.
14.Create Virtual hosts.
15.Max and minimum heap size.
16.Debug argument.
17.Http Transports
18.Boot classpath.
19.Classpath.
20.Create server level variable.
21.JVM arguments.
22.Changes ports of server.
23.JDBC and Data source creation.
24.Create JDBC provider at cluster level scope..
25.Create Data Source at cluster level scope.
26.Create J2c Authentication.
27.Create Connection Pool Setting.
28.Testing Database connection.
29.Cluster Stop and start.
30.Application installation.

Pre-requisites:-


These are the pre-requisites which we need to do before installing this tool.

1. Create one user which will be used as administrator.
2. ssh for this user should work in all UNIX boxes.
3. Directory under which deployment manager, node and webserver are to be installed should be under this user.
4. The Directory under which we will place this tool should be mounted on all UNIX boxes where we will be installing Websphere Application server.
5. Download the Websphere application server and HTTP server binaries.

Description:-

This tools does Base installation of Websphere Application server version 6.1, first it checks whether the WAS base installation is already done on UNIX m/c and if the base installation is not there it will install WAS base binaries. The location under which it will check whether WAS is already installed depends on the value specified in configure.properties file and response file and then it install Update installer and apply fix packs and after installing base binaries, it install deployment manager using manageprofile utility and values specified in property files and using ports defined in ports properties file. It also checks whether the ports are already in use or not if they are in use I will send the list the ports which are already in use and it also changed the SOAP TIME OUT value to 6000 as it is 180 by default which is very less.

It will then install number of Nodes specified in property file and install them on host specified in properties file and install the Base binaries if they are not installed on unix box and it they are installed then it install nodes and federate nodes to deployment manager, changes node agents ports and start node. It also changes Soap time out for nodes.
*It also install node and federate with Deployment Manager for existing environment.

After node installation is completed, then it installs webservers on the host where we want to install it as specified in property files and configure webserver to deployment manager and generate plug-in and propagate the plug-in back to webserver and start the admin and http server. Http server will be listening on the port in property file.



It then create cluster and create server and add server to cluster and configure all these things on server:-
o Create Virtual hosts.
o Max and minimum heap size.
o Web Container Thread Pool Setting
o Debug argument.
o Http Transports
o Boot class path.
o Class path.
o Create server level variable.
o JVM arguments.
o Changes ports of server.

All these values are picked from property files.
• Other things can also be configured depending upon requirement and script can be modified to accommodate those changes.

After configuring cluster and server creation now it create JDBC provider, Data source , J2c Authentication and connection pool setting on cluster level scope.

It then installs the application on cluster which is created and starts the cluster and sends an email out with Deployment manager console login.

So with one script execution it will do a full fledged environment creation taking all possible intervention of manual stuff to be done and script which we will have to execute will be WASIC.sh.

All these steps in environment creation can run as on single step and they can be executed separately as single steps depending upon needs and new feature can be added as per requirement.


NOTE ***** ALL VALUE ARE PULLED FROM PROPERTY FILES, SO EVERYTHING CAN BE VERSION CONTROLLED IN CLEARCASE.


This tool also works under these scenarios also:

1. If we have to just create Deployment manager.
2. If we have create standalones Application server.
3. If we have to add new node to existing environment.
4. If we have to add new cluster in existing environment.
5. If we have to add new server to existing cluster and configure it.
6. If we have to modify existing JDBC, Data source connection settings.
7. For daily to daily application deployments, cluster stop and cluster start.
8. Create new webserver.
9. Create new webserver and add this webserver to existing environment.
10. It will generate plug-in after application installation or any configuration change and propagate the plug-in to webserver.

For all this environment creation to work, all we have to do is copy the existing environment in WASIC configuration and rename it to the new environment what we want to create change the values in properties file and change the name of scripts and run the WASIC.sh with functional area name and environment name.

These all steps can also be integrated as part of Build Forge.

FUTURE ENHANCEMENT:

1. Create a Graphical user interface for WASIC tool.
2. Adding New features for V 7.0 which are installation of admin agent, Job Manager and registering node to admin Agent and job manager
3. Configuring Global Security, LDAP, LTPA, SSO.
4. Configuring SSL configuration, creating self signed certificate, replace an existing self signed certificate, creating certificate authority requests, receiving a certificate issued by a certificate authority, retrieving a signer certificate from a remote ssl port and adding a signer certificate to a keystore.
5. Installations of Websphere Portal Sever and Websphere Process server.
6. This tool can be modified as per requirement and more features can be added.
7. These scripts can be broken into small scripts, so that if we have to change one configuration file of server it will work.
8. The scripts can also be integrated with build Forge

Release management

Release management defines the mechanisms of building and releasing software, and is included as a component of the Service Support Set in ITIL. The practice of Release Management continues to evolve while being applied to complex distributed software services such as in the SOA realm.

Release Management is proactive technical support focused on the planning and preparation of new services. Some of the benefits are:-

* The opportunity to plan expenditure and resource requirements in advance.
* A structured approach to rolling out all new software or hardware, which are efficient and effective.
* Changes to software are ‘bundled’ together for one release, which minimises the impact of changes on users.
* Testing before rollout, which minimises incidents affecting users and requires less reactive support.

The process

The release management meta process consists of several steps.

Gathering and description

When a new release is prepared, requirements are gathered. These are for example improvements that are needed to fix the current product. A parallel step is to look at the dependencies. Programs often consist of many modules that depend on each other to work. Changing one will affect the other. Once the requirements and dependencies are known, the next release process can be planned. This planning consists of what steps need to be taken, the time constraints etc. In figure 1 the entire process is visualized, using the meta modeling technique.

Many IT shops, especially those with extensive Microsoft platform deployments have developed elaborate processes for Patch Management to the Production environment. Their scope usually includes operating systems software, database upgrade, and even firmware upgrades to hardware components (e.g. storage arrays and network switches).

Release Building

When the requirements, dependencies and planning are known, the building process of the new release begins. The first step is to design the new release. This can be done with the use of various software development techniques for example UML. The design is worked out into code in a programming language (e.g. Java, C#, C++). The pieces of code, classes etc. are joined together, compiled into working subsections, and finally put together in a working program, a build.

Acceptance Test

When the build is ready, it is sent to a testing department for further acceptance testing, checking the build against the testing standards. The program is reviewed to verify if it works correctly and lives up to the requirements and dependencies. During this time the entire process is documented to serve as a future knowledge base. After a final verification of the program the testing standards are updated.

Release Preparation

Once a correct, verified release has been achieved, it is prepared for release to the production environment. The release is packaged, meaning preparing a final product to be sent to a specific customer. This can be an Internet download, a CD, or a specific language etc. A final step is the verification of this package against the requirements resulting in audit reports. These audit reports give a last verification before the entire package is released.

Release Deployment

The deployment itself is getting the release to the customer and implementing it.

Software development process

A software development process is a structure imposed on the development of a software product. Synonyms include software life cycle and software process. There are several models for such processes, each describing approaches to a variety of tasks or activities that take place during the process.

Software development activities

Planning

The important task in creating a software product is extracting the requirements or requirements analysis. Customers typically have an abstract idea of what they want as an end result, but not what software should do. Incomplete, ambiguous, or even contradictory requirements are recognized by skilled and experienced software engineers at this point. Frequently demonstrating live code may help reduce the risk that the requirements are incorrect.

Once the general requirements are gleaned from the client, an analysis of the scope of the development should be determined and clearly stated. This is often called a scope document.

Certain functionality may be out of scope of the project as a function of cost or as a result of unclear requirements at the start of development. If the development is done externally, this document can be considered a legal document so that if there are ever disputes, any ambiguity of what was promised to the client can be clarified.

Design

Domain Analysis is often the first step in attempting to design a new piece of software, whether it be an addition to an existing software, a new application, a new subsystem or a whole new system. Assuming that the developers (including the analysts) are not sufficiently knowledgeable in the subject area of the new software, the first task is to investigate the so-called "domain" of the software. The more knowledgeable they are about the domain already, the less work required. Another objective of this work is to make the analysts, who will later try to elicit and gather the requirements from the area experts, speak with them in the domain's own terminology, facilitating a better understanding of what is being said by these experts. If the analyst does not use the proper terminology it is likely that they will not be taken seriously, thus this phase is an important prelude to extracting and gathering the requirements. If an analyst hasn't done the appropriate work confusion may ensue: "I know you believe you understood what you think I said, but I am not sure you realize what you heard is not what I meant."[1]

Specification

Specification is the task of precisely describing the software to be written, possibly in a rigorous way. In practice, most successful specifications are written to understand and fine-tune applications that were already well-developed, although safety-critical software systems are often carefully specified prior to application development. Specifications are most important for external interfaces that must remain stable. A good way to determine whether the specifications are sufficiently precise is to have a third party review the documents making sure that the requirements and Use Cases(A use case in software engineering and systems engineering is a description of a system’s behavior as it responds to a request that originates from outside of that system) are logically sound.

Architecture

The architecture of a software system or software architecture refers to an abstract representation of that system. Architecture is concerned with making sure the software system will meet the requirements of the product, as well as ensuring that future requirements can be addressed. The architecture step also addresses interfaces between the software system and other software products, as well as the underlying hardware or the host operating system.

Detailed Design

The Detailed Design is kind of the translation of architecture, which is a concrete operational guide following architecture. It may cover programming specific desgns, UI and validations, database structure, and also embed design pattern and best practice.

Implementation, testing and documenting

Implementation is the part of the process where software engineers actually program the code for the project.

Software testing is an integral and important part of the software development process. This part of the process ensures that bugs are recognized as early as possible.

Documenting the internal design of software for the purpose of future maintenance and enhancement is done throughout development. This may also include the authoring of an API, be it external or internal.

Deployment and maintenance

Deployment starts after the code is appropriately tested, is approved for release and sold or otherwise distributed into a production environment.

Software Training and Support is important because a large percentage of software projects fail because the developers fail to realize that it doesn't matter how much time and planning a development team puts into creating software if nobody in an organization ends up using it. People are often resistant to change and avoid venturing into an unfamiliar area, so as a part of the deployment phase, it is very important to have training classes for new clients of your software.

Maintenance and enhancing software to cope with newly discovered problems or new requirements can take far more time than the initial development of the software. It may be necessary to add code that does not fit the original design to correct an unforeseen problem or it may be that a customer is requesting more functionality and code can be added to accommodate their requests. It is during this phase that customer calls come in and you see whether your testing was extensive enough to uncover the problems before customers do. If the labor cost of the maintenance phase exceeds 25% of the prior-phases' labor cost, then it is likely that the overall quality, of at least one prior phase, is poor. In that case, management should consider the option of rebuilding the system (or portions) before maintenance cost is out of control.

Bug Tracking System tools are often deployed at this stage of the process to allow development teams to interface with customer/field teams testing the software to identify any real or perceived issues. These software tools, both open source and commercially licensed, provide a customizable process to acquire, review, acknowledge, and respond to reported issues.

Build Automation

Build automation is the act of scripting or automating a wide variety of tasks that software developers do in their day-to-day activities including things like:

* compiling computer source code into binary code
* packaging binary code
* running tests
* deployment to production systems
* creating documentation and or release notes

Types

* On-Demand automation such as a user running a script at the command line
* Scheduled automation such as a continuous integration server running a nightly build
* Triggered automation such as a continuous integration server running a build on every commit to a version control system

Advantages

* Improve product quality
* Accelerate the compile and link processing
* Eliminate redundant tasks
* Minimize "bad builds"
* Eliminate dependencies on key personnel
* Have history of builds and releases in order to investigate issues
* Save time and money - because of the reasons listed above.[6]


Requirements of a build system

Basic requirements:

1. Frequent or overnight builds to catch problems early.
2. Support for Source Code Dependency Management
3. Incremental build processing
4. Reporting that traces source to binary matching
5. Build acceleration
6. Extraction and reporting on build compile and link usage

Optional requirements:

1. Generate release notes and other documentation such as help pages
2. Build status reporting
3. Test pass or fail reporting
4. Summary of the features added/modified/deleted with each new build

Script to check the status of application and send email notification if application is down

This Script checks the status of application and send email notification if application is down. Define the application URL in url.properties
#!/usr/bin/bash

## Written by Charanjeet Singh ##

PATH=${PATH}:/opt/csw/bin;export PATH

cd /home/charan
date=`date '+%Y%m%d%H%M'`
mdate=`date '+%b-%d(%A)-%Y_%H.%M%p'`
home=/home/charan
logdir=$home/logs/application/$1.app.log$date
emaillist=
if [ $# -ne 1 ] ;
then echo ;
echo "enter env name for the script to execute or pass ALL as argument to check for the status of all envrionment specified in url.properties"
echo
exit 0
fi
echo " Checking application status of $1 Server "
envlist=" DEV SIT UAT "
if [[ $1 == ALL ]]; then
for envvar in $envlist ;
do
PropsPath="/home/ccbuild/charan/url.properties";
URL=`grep "$envvar"= $PropsPath | awk -F= '{print $2}'`
echo " $envvar URL --> $URL "
wget -q -O $logdir $URL
if [ \! -s $logdir ]; then
mailx -s "$envvar Application is DOWN" $emaillist < $logdir rm -rf $logdir fi done fi for env in $envlist ; do if [[ $env == $1 ]]; then PropsPath="/home/charan/url.properties"; URL=`grep "$env"= $PropsPath | awk -F= '{print $2}'` echo " $env URL --> $URL "
wget -q -O $logdir $URL
if [ \! -s $logdir ]; then
mailx -s "$env Application is DOWN" $emaillist < $logdir
rm -rf $logdir
fi
fi
done
rm -rf $logdir

--------------------------------------------------------------

DEV=
SIT=
UAT=

Automation Using Scripting

There are Number of ways in which we can automate tasks on Windows .Net and Unix Platforms:-

1. Power Shell
2. Batch File
3. WMI(Windows Management Instrumentation)
4. Nant
5. Ant
6. Perl
7. Shell Scripting.
8. VB script.
9. System administration automation