Friday 13 June 2014

Get Parent Directory via Python

Get the parent directory of a folder/file in python. Using the "os" library its possible using python.


Lets try to understand with an example

>>> import os
>>> os.path.dirname('D:\Workspace\8.0.0.0 - INT')

Output will be as: 'D:\\Workspace'

Same if want to get the parent directory of file,

>>> os.path.dirname('D:\Workspace\8.0.0.0 - INT\AdapterFactory.py')
Out put will be as: 'D:\\Workspace\\8.0.0.0 - INT'

Thanks.

Merge Two CSV Into One CSV

Hello my so lovely friends,

So how are your days going on? Hope you guys doing some experimenting. NOOO.. not an issue.

During my office hours I met with a problem to merge two CSV's into one on the basis of the common header, no need to include those records with mis match headers with another CSV. After experimenting finally able to write a script in python. So here is the code for the same. Hope if any one is facing or will face same issue can use this script.
Link for the script: https://gist.github.com/anupamshakya7/60607c86a7dbe9853843

So here is the script. Enjoy my friends and have a good day :)

import csv
import os

if __name__ == '__main__':

    """This script will merge two CSV with common data as per the ID as a field
    into new CSV.

    ::NOTE:: This program tested on 2.7.6 version of python.

    For an example Test1.csv have this data:
    ----------------------------------------
    ID,name,add,school
    1,anupam,FHG,sch1
    2,aditya,DFR,sch2
    3,urmila,HJTY,sch3
    4,roop,GHG
    5,EASH,HJJ

    Test2.csv have this data:
    ------------------------
    ID,Key,Value
    1,x-jdkj,100
    2,k-djsh,200
    3,j-jdjd,300

    Resultant CSV have this data:
    -----------------------------
    add,school,ID,name,Value,Key
    FHG,sch1,1,anupam,100,x-jdkj
    HJTY,sch3,3,urmila,300,j-jdjd
    DFR,sch2,2,aditya,200,k-djsh


    How to run
    ----------
    On command promt go to the location where you placed your script(python script)
    And issue this command 'python CSV.py'

    After that it will ask for the absolute path where you have placed your two CSV with ID
    as a common field(in header part).

    ::NOTE:: Please provide absolute path for folder like - C:\Users\hv3775\Desktop\TEST

    That's. It will generate new CSV into the same folder that you have provided as a input path.
    """

    input_folder_location = raw_input('Enter location where your required two csv files placed. NOTE:: Please enter absolute path: ')
    infiles = []
   
    for csv_file in os.listdir(input_folder_location):
        if csv_file.endswith('.csv'):
            infiles.append(csv_file)
   
    data = {}
    fields = []

    for fname in infiles:
        with open(fname, 'rb') as df:
            reader = csv.DictReader(df)
            for line in reader:
                # assuming the field is called ID
                if line['ID'] not in data:
                    data[line['ID']] = line
                else:
                    for k,v in line.iteritems():
                        if k not in data[line['ID']]:
                            data[line['ID']][k] = v
                for k in line.iterkeys():
                    if k not in fields:
                        fields.append(k)
            del reader

    data_list = []
   
    for d in data.items():
        if len(d[1].values()) != len(fields):
            continue
        data_list.append(d[1])  
   
    csv_output_location = os.path.join(input_folder_location, 'Result.csv')
    with open(csv_output_location, 'wb') as f:
        w = csv.DictWriter(f, fields)
        w.writeheader()
        w.writerows(data_list)


Wednesday 11 June 2014

Very very thanks


Want to thanks all those who visited my blog for their respective quires. No views of my blog is more than 5000. Its all due to you my friends. Please share your suggestions or want to see any improvement.

Saturday 7 June 2014

Difference Between Compiler and Interpreter

A Compiler and Interpreter both carry out the same purpose – convert a high level language (like C, Java) instructions into the binary form which is understandable by computer hardware. Specific compilers/interpreters are designed for different high level languages. However both compiler and interpreter have the same goal but they different in the way they accomplish their task i.e. convert high level language into machine language. Through this article we will talk about the basic knowledge of both and distinguish the basic difference between compiler and interpreter.
Compiler
Compiler is a piece of code that translates the high level language into machine language. When a user writes a code in a high level language and wants to execute, a specific compiler which is designed for that specific language is used before it will be executed. Compiler scans whole program first and then translates it into machine code which will be executed by the computer processor and the corresponding tasks will be performed.  
Here program written in higher level language is known as source code and converted one is called object program.

Interpreter
Interpreters are not much different than compilers. They also convert the high level language into machine readable binary equivalents. Each time when an interpreter gets a high level language code to be executed, it converts the code into an intermediate code before converting it into the machine code. Each part of the code is interpreted and then execute separately in a sequence and an error is found in a part of the code it will stop the interpretation of the code without translating the next set of the codes.  
In the above figure, first a source code is converted to an intermediate form and then that is executed by the interpreter.
The main differences between compiler and interpreter are listed below:
  • The interpreter takes one statement then translates it and executes it and then takes another statement. While the compiler translates the entire program in one go and then executes it.
  • Compiler generates the error report after the translation of the entire page while an interpreter will stop the translation after it gets the first error.
  • ·Compiler takes a larger amount of time in analyzing and processing the high level language code comparatively interpreter takes lesser time in the same process.
  • The compiler derives its name from the way it works, looking at the entire piece of source code and collecting and reorganizing the instructions. Thus, a compiler differs from an interpreter, which analyzes and executes each line of source code in succession, without looking at the entire program. 
  • Besides the processing and analyzing time the overall execution time of a code is faster for compiler relative to the interpreter.

Oracle Dual Table

I'm very new to oracle, so as of now no knowledge, but learning as I've to survive in IT industry :P. NO other my dear friends. During my learning I met with a term "Dual Table". So whatever I learned
going to share with you. Hope will help a pro who is new to this field just like me :) 

Oracle Dual Table is a table which is created with the data dictionary. Consists exactly one column whose name is dummy and one record. The value of that record is X.

Run following  query on Oracle SQLDeveloper
desc dual, You will this result
Name                    Null?    Type
----------------------- -------- ----------------
DUMMY                            VARCHAR2(1)

Now hit select query on this dual table.
select * from dual; Result as

D
-
X

The owner of dual is SYS but dual can be accessed by every user.
As dual contains exactly one row (unless someone fiddled with it), it is guaranteed to return exactly one row in select statements. Therefor, dual is the prefered table to select a pseudo column (such as sysdate
select sysdate from dual
Although it is possible to delete the one record, or insert additional records, one really should not do that!.

If this article helped you, don't forget to share it with others.
Thanks.

Friday 30 May 2014

Get list of installed libraries in a system using python

From lovable python terminal its possible to get list of all installed python libraries. Here is the piece of cake for you that I already tested. Try from yourself too :)

Open your python terminal doesn't matter whatever OS you are using(As of now I' using windows. Type following command

from pkg_resources import WorkingSet , DistributionNotFound
working_set = WorkingSet()

# Printing all installed modules
print tuple(working_set)

# Detecting if module is installed
try:
    dep = working_set.require('paramiko>=1.0')
except DistributionNotFound:
    pass

Output of above print will be as:
(astroid 1.0.1 (c:\python27\lib\site-packages), BeautifulSoup 3.2.1 (c:\python27\lib\site-packages), blinker 1.3 (c:\python27\lib\site-packages),....


Thursday 24 April 2014

From CMD change directory from C to D

Friend I'm back. From last 3 years fall in love with python only, so forgot windows..LOLZZZ....

While using windows facing an issue to change the directory from C drive to D drive. But found solution for the same. Lets explain via an example. Here we go.....

Lets consider we are in C drive
C:\Users\hv3775>cd /d "D:\"

To go to D drive like this D:\, type following command on cmd and you are done :)

C:\>cd /d "D:\"

Thanku

Transform an XML file using XSLT in Python

Howdy,

Form last two months was so busy, so not able to write any new post. But in last two months while doing my project encountered with an issue. The issue was to generate CSV files from the given XML & XSLT file using python. But by doing some research able to find solution using the most popular library LXML. Lets try to explore how its is possible.


Go to your python terminal, and import lxml library
>>> from lxml import etree

1) Open xslt file and read its content.
>>> data = open('D:\Conversion\MACSXML_Parts.xslt')
>>> xslt_content = data.read()

2) Now parse xslt content to etree.XML method. "etree.XML" returns root node (or the result returned by a parser target).
>>> xslt_root = etree.XML(xslt_content)

3) Parse required XML file to etree.parse method, will return ElementTree object like this "<lxml.etree._ElementTree object at 0x01AB1F08>"
>>> dom = etree.parse('D:\Conversion\Cat2015UF.xml')

4) Parse xslt_root node to etree.XSLT(created above). "etree.XSLT" turn an XSL document into an XSLT object.
>>> transform = etree.XSLT(xslt_root)

5) Finally parse dom that has been created above to the new created transformed XSLT object.
>>> result = transform(dom)

Your result contained the appropriate data. You can store this into CSV file like:
>>> f = open('D:\Conversion\MACSXML_Parts.csv', 'w')
>>> f.write(str(result))
>>> f.close()

Full example is as:


 from lxml import etree

 data = open('D:\Conversion\MACSXML_Parts.xslt')

 xslt_content = data.read()
 xslt_root = etree.XML(xslt_content)
 dom = etree.parse('D:\Conversion\Cat2015UF.xml')
 transform = etree.XSLT(xslt_root)
 result = transform(dom)
 f = open('D:\Conversion\MACSXML_Parts.csv', 'w')
 f.write(str(result1))
 f.close()
You can view the same code on gist too. Link is as https://gist.github.com/roopsinghadi/11285898
Hope it is helpful. If it's don't forget to share :P

Tuesday 1 April 2014

Transfer files or folder from windows to Unix using putty Or vice versa

Sometimes we want to transfer files/folder from UNIX server to our windows via putty. So its possible??

Its possible via pscp. Wait what is the pscp?? :P

"Its a tool for transferring files from computers to SSh or vice versa."
Picking one scenario to use pscp. Lets consider I want to transfer one file from windows to unix, So the command for the same is given below. You have to run this command on cmd

Main syntax is:
" pscp [options] source [source...] [user@]host:target "

Command for the above scenario is as:

C:\Program Files\PuTTY> pscp C:\Test.xml  hv3775@sbsdev01:/A/GM/app/script/TestFolder/

Explanation of above example:

  • pscp - The PuTTY Secure Copy client, is a tool for transferring files securely between computers using an SSH connection.
  • C:\Cat2014ZD.xml  - Its the source, which we want to transfer.
  • hv3775@sbsdev01:/A/GM/app/script/TestFolder/ - It is the user+host+path where we want to move files.







References:
http://the.earth.li/~sgtatham/putty/0.60/htmldoc/Chapter5.html
http://www.it.cornell.edu/services/managed_servers/howto/file_transfer/fileputty.cfm
https://community.freescale.com/thread/220596

Friday 28 February 2014

Get the Content-Type of an URL

Hey everybody,
Have you ever tried to get the content type of any website via python, If not try like this. I tried on python terminal.
OR
>>> import urllib
>>> res = urllib.urlopen("https://www.wslb2b.ford.com/login.cgi")
>>> http_message = res.info()
>>> http_message.type
'text/html'
>>> http_message.maintype
'text'
Thanku :)