Introduction to DSCop

DSCop is an open source tool that analyzes IBM InfoSphere DataStage jobs and reports information such as violation of some commonly accepted best practices. It's developed in C# and provides plugin based architecture to allow 3rd party extensibility. The tool comes with a few sample plugins that should be enough for basic understanding how the tool works and how to implement your own plugins.

Requirements

Computer running version of Microsoft Windows with .NET Framework 4 preinstalled.

Download and Install

The current publicly available version of DSCop is RC1 available here. After download unzip the file and you are ready to use the tool.

Basic Usage Scenario

The tool will automatically search the folder where the DSCop.exe file is located and will load all the plugins. Each plugin contains one or more rules. Each rule enforces certain check that is performed on  a DataStage job. The jobs should be exported to one or many xml files.

To check a job with the tool, use this syntax:

DSCop jobfilename.xml

As a result you will get a list with rules that were executed, jobs processed, rule violations found.

A sample output after running the tool is shown here:

DataStage has the ability to export multiple jobs into singe XML file. However if your jobs are not in one file you can use wildcards to specify them like this:

DSCop Staging*.xml

Advanced Usage Scenario

Now we are going to explore some of the advanced use cases where you want to run the tool by including/excluding certain rules.

The syntax for running the tool by explicitly enumerating the rules you want to execute is:

DSCop jobfile.xml –include RuleName1 RuleName2 RuleName3

A sample output using this syntax is shown bellow:

Please notice the “*Ignore” next to the rules that are not enforced.

To exclude certain rules use the following syntax:

DSCop jobfile.xml –exclude RileName1, RuleName2, RuleName3

Sample Plugins

The RC1 version of the tool comes with the following plugins/rules:

• CoreRules/StableSortRule – checks all Sort stages whether the StableSort=true. StableSort is enabled by default but should not be used due to decreased performance.
• CoreRules/TeradataConnectorParametersRule – checks all Teradata connector stages whether the ServerName/Username/Password properties are parameterized. Hardcoding this information should be avoided.
• NamingRules/PrefixNamingRule – checks if stage names are following predefined naming convention. The naming convention is described in a the file NamingPrefixes.xml located in the same folder as the plugin.

Feedback

You can send your feedback to the following email: dscoptool-at-gmail.com

Some tips for the SSD owners running Windows

Solid State Drives (SSD) are much different than Hard Disk Drives (HDD). Because of this you must change your usual pattern of usage. Otherwise you risk your shiny new SSD to be at the end of its life only after a few months of use. In this post I will describe some of the tricks that seems to work on my machine.

The main disadvantage of the SSD devices is that each data cell can be re-written limited number of times. This number is several times lower than the data cell in the HDD. So you must make sure that the applications you are using are not constantly writing on the SSD. Bellow is a short list that will extend the life of your SSD.

Must Have

 

Disable Windows Swap File

If Windows runs out of RAM then it moves some of the data to the swap file on your disk drive. This is one of the primary sources of disk write operations on systems running with low RAM memory. Buy more RAM and disable Windows swap file. The RAM you need depends on the applications you are running and the version of Windows. IMHO for Windows 7 you need at least 4GB RAM, 8GB is recommended.

Turn off Disk Defragmenter Schedule

The classic hard disk drives are using a moving head that reads the data from the disk. If the file is not located on sequential blocks on the disk, the head positioning time will increase and this will slow down the read process. The defragmentation process makes sure that the file blocks are located in a sequence on the disk. The SSD do not suffer from this since there is no head that reads the information. So you don’t need this feature.

If you perform defragmentation on SSD drive this will only drain write cycles from its life.

Windows 7 automatically detects SSD drives and turns of the defragmentation but it does not hurt to check if it has been stopped. On Windows 7 you can disable the Disk Defragmenter Schedule like this.

Move Temp Folders to RAM Drive

RAM drive is a disk drive that looks looks like a normal drive for your windows but stores the information in the RAM instead of using a SSD or HDD. The operations with RAM drive are several times faster than the operations with SSD or HDD. The downside is that you lose the drive content after powering off your computer. But the Temp folder is used by Windows to store temporary files not needed after power off. So you don’t have to worry that you will lose something important.

Moving the windows Temp folder to a RAM drive will decrease the write operations on your SSD and this prolongs its life. Here is how to move the Windows Temp folder to another location. The only think you should select is the software that emulates your RAM drive since Windows still does not have such a feature. For example the QSoft’s RAMDisk does a perfect job for me. You can download a free version that will expire after 6 months (at which point you need to download new free version which will be active for another 6 months).

You can even move the location of Internet Temp folder but this will slow down your browsing since there will be no cached images or pages and everything will be downloaded again after you power off the PC.

Turn off Windows Search

Windows Search is another feature that generates a lot of write operations due to the indexing process. The SSD are quite fast on reading data so you don’t need indexes to speed up your search. You can turn it off like this on Windows 7 systems. Of course if you are not satisfied you can always turn it on.

Monitor the Overall Health of the Drive

Install an application that will report your drive health. For example a good choice is the free version of SSDLife. It can be downloaded here. The most useful metric is the approximate date when the drive is supposed to failure.

 

Nice To Have

 

Hibernation

Hibernation becomes very tempting when combined with fast disk drive. The down side is that this feature stores GB of information (depending on you RAM size) every time when the system is hibernated. Having in mind that the SSD drives are sensible to the number of writes it may quickly drain the life from your new SSD drive. And of course after disabling you get some more free space on your drive. Here is how to do it on Windows 7.

Disable System Restore

System Restore is helpful if you mess up your system – for example installing wrong device driver. However, this feature consumes significant amount of free space on your SSD drive. If you are suffering from free space problems and you are confident that changes you make to your system are safe you can disable System Restore.