0 votes
in Python by Beginner (2 points)
Hello all,

If both CSV files are having the same columns order and number of columns then it will be easy to do comparison, But in my case columns order and number of columns also will change.

Could you please help to compare 2 different csv files data and generate new csv file with the result of the comparison in Python.

Thank you!

2 Answers

0 votes
by Beginner (4 points)
reshown by

This isn't a Python solution but maybe it will help [line edited by wellsr]

If the columns are not ins the same order, there is nothing that will allow you to make valid comparisons

Is there any sort of logic to the way the columns are disorganzied?

Can you get your source to provide them in a standard way?

Are you getting them from more than one source, so they always come from the same source in the same column order?

How do you want to compare them? on specific cells or the values in whole row?

What do you want to do with the comparison? Eliminate duplicates? Just identify duplicates,

If you can define some consistency you should be able to use PowerQuery to do everything you need. PowerQuery tracks what you do when you import a file, then you can re-use those steps when you get more data. 

For example, if you are getting files with the same data, in different column order, from 3 different places, you can define separate queries for each location to get the data all in the same format. Then you can set up a fourth query to look for duplicate rows and do "something" with them.

Here are some articles with examples of that process. Is taht something you want to go ahead with?

Getting Started with “PowerQuery” / “Get & Transform” > Tables > PivotTables > PivotCharts > Dashboards ... PowerBI
WIKI:
   https://answers.microsoft.com/en-us/msoffice/forum/msoffice_excel-mso_win10-mso_o365b/get-started-with-get-transform-tables-pivottables/fea2f31d-7835-4f77-ad34-6fe3e6d07620?tm=1537369871055

Microsoft Power Query for Excel Help  (in wiki)
https://support.office.com/en-us/article/microsoft-power-query-for-excel-help-2b433a85-ddfb-420b-9cda-fe0e60b82a94
This is MS home page for PowerQuery help, with links to MANY detailed help pages
Power Query provides data discovery, data transformation and enrichment for the desktop to the cloud.
Easily discover, combine, and refine data for better analysis in Excel.
.  *  Introduction to Microsoft Power Query for Excel
.  *  Import data from external data sources
.  *  Shape data from multiple data sources

 

You may be even be able set PowerQuery up to extract the data directly from other corporate databases:

(Over 80) External Data sources available in Power Query. (in Wiki)
https://d13ot9o61jdzpp.cloudfront.net/images/pq_get_data.png
https://www.myonlinetraininghub.com/power-query-and-power-pivot-definitive-guide

 

Combine CSV Files with Power Query
https://exceleratorbi.com.au/combine-csv-files-power-query/
CSV files are relatively simple to combine provided they are in the same format and are generally tabular in layout.


 

Combine multiple files of different file types            2015 01 15
https://powerpivotpro.com/2015/01/power-query-for-excel-combine-multiple-files-of-different-file-types/
Power Query Magic:  The Ultimate and easiest way to consolidate multiple tables, sheets, text and/or csv files
At some point in the life of an Excel user, we have all faced a similar dillemma. How can I combine multiple sheets, tables, csv or txt files? (can I combine them all together??)
How we used to solve this scenario
Back in the day (before Power Query) we actually had some ways to do so but they were not so user-friendly and they relied heavily on coding or some tedious way of doing it.

 

0 votes
by Beginner (25 points)

I think you can using sub CompareColumns (Author:MR Wellsr) on this site.

Welcome to wellsr Q&A
wellsr Q&A is the VBA and Python programming community that rewards you for learning how to code.

Getting Started
Register
VBA Cheat Sheets (On Sale Now)

Earn free prizes for asking VBA and Python questions and for answering questions asked by others in our community.

Looking for something else? Hire our professional VBA Help, instead.

What makes us different?
Our points system rewards you with a chance for free gifts based on the quality of your questions and answers. All you have to do is post and you could get rewarded, like these members:

ParserMonster $25 Amazon Gift Card
Hightree $10 Amazon Gift Card
Thales1 $10 Amazon Gift Card
runfunke $10 Amazon Gift Card
coolag $10 Amazon Gift Card
Siew Hun $10 Amazon Gift Card

So, why don't you join us? It really is an encouraging way to motivate members in our VBA and Python community.

Register

For more programming tips visit the VBA Tutorials Blog and the Python Tutorials Blog.

...