Latest Tutorials| Questions and Answers|Ask Questions?|Site Map



Home Answers Viewqa Development-process is there any possibelities fast read and write file large data file

Have Programming Question? Ask it here!
 
 
 


muralanagaraju
is there any possibelities fast read and write file large data file
0 Answer(s)      a year and 5 months ago
Posted in : Development process


import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.util.Hashtable;
import java.util.Vector;
public class Groupingclass {

public void prepareGroupBy(Vector<Integer> vec,String delimeter) {
    BufferedReader readoldfile = null;
    //BufferedWriter bufferFileWriter  = null;
        try{
            String sCurrentLine;
            // Groped file name (Suresh.txt)
            readoldfile = new BufferedReader(new FileReader("D:\\suresh.txt"));
            int count =0;
            while ((sCurrentLine = readoldfile.readLine()) != null) {
                if(count ==0){
                    groupByDataWriteToFile(sCurrentLine+delimeter+"1","first",vec ,delimeter);
                }
                else {                  
                    if(sCurrentLine !=null && sCurrentLine.toString().trim().length()>0)
                    {
                        /*Hashtable<String,Object> resultHash =*/ groupByDataWriteToFile(sCurrentLine+delimeter+"1","same",vec,delimeter);
                    /*
                    if(resultHash!=null && resultHash.size()>0) {
                         Boolean status = (Boolean) resultHash.get("isexist");
                         if(!status){
                             groupByDataWriteToFile(sCurrentLine+delimeter+"1","unique",vec,delimeter );
                         }
                     }*/
                    }
                } 
                count++;                
            }
        } catch(Exception ex){
            //File file = new File("D:\\groupby.txt");
            System.out.println("Exception>>>????????????????"+ex);
        }
        finally {
            try {
            readoldfile.close();
            //bufferFileWriter.close();
            }catch(IOException io){}
        }
}
public  Hashtable<String,Object >  groupByDataWriteToFile(String oldCurrentLine,String unique,Vector<Integer> vec,String delimeter) throws IOException
{
    Hashtable<String,Object> resultHash = new Hashtable<String,Object>();

    BufferedReader br = null;
    BufferedWriter bufferFileWriter  = null;
    boolean isexist = false;
    File path = new File("D:\\groupBy.txt");
    int countR=0;
    try {
          String sCurrentLine;
         String writepath = "D:\\groupBy.txt";     
         File file = new File(writepath);     
         bufferFileWriter = new BufferedWriter(new FileWriter(file,true));  

         if(path.exists()) {
                br = new BufferedReader(new FileReader(path));
         }
         else {
                    File gfile = new File("D:\\groupBy.txt");
                    br = new BufferedReader(new FileReader(gfile));
             }           
         if(unique !=null && unique.equals("first")) {
            // System.out.println("firs+++++tline :"+oldCurrentLine);
             bufferFileWriter.write(oldCurrentLine);
                bufferFileWriter.newLine();
                bufferFileWriter.flush();
                isexist = true;
                resultHash.put("isexist",true);
                //resultHash.put("currentRow",oldCurrentLine);
                return resultHash;
         }
         else {
             while ((sCurrentLine = br.readLine()) != null) {
                 //read from start original file
                 String arry[] = sCurrentLine.split(delimeter); 
                // it take row by row from original file
                 String oldarry[]  =oldCurrentLine.split(delimeter); 
                 boolean groupBycolumnsCheckFlag = checkGroupByColumns(vec,arry,oldarry);
                // if row is same true
                 if(groupBycolumnsCheckFlag) {
                        String cnt= arry[(arry.length-1)];
                        countR =Integer.parseInt(cnt);
                        countR = countR+1;
                        arry[(arry.length-1)] =countR+"";

                        String finalStr = UpdateSameRowWithCount(vec,oldarry,arry);
                        // System.out.println("finalStr :"+finalStr);
                        bufferFileWriter.write(finalStr);
                        bufferFileWriter.newLine();
                        bufferFileWriter.flush();
                        isexist = true;
                        resultHash.put("isexist",true);
                        resultHash.put("currentRow",sCurrentLine);
                        closeConnections(br,bufferFileWriter);
                        if(path.exists()) {
                        temFileCreate(new BufferedReader(new FileReader("D:\\groupBy.txt")),oldCurrentLine,countR,vec,delimeter);
                        }
                        else {
                            File gfile = new File("D:\\groupBy.txt");
                            temFileCreate(new BufferedReader(new FileReader(gfile)),oldCurrentLine,countR,vec,delimeter);
                        }
                        resultHash.put("isexist",true);
                        resultHash.put("currentRow",sCurrentLine);
                        return resultHash;
                 }
                 else if(!sCurrentLine.equals(oldCurrentLine)) {
                        resultHash.put("isexist",isexist);
                        resultHash.put("currentRow",sCurrentLine);
                    }
             }          
         }
    // System.out.println("not same :"+oldCurrentLine);
         if(file.exists()) {
            bufferFileWriter.write(oldCurrentLine);
            bufferFileWriter.newLine();
            bufferFileWriter.flush();
            resultHash.put("isexist",true);
            resultHash.put("currentRow",oldCurrentLine);
         }
         else{
             File file1 = new File("D:\\groupBy.txt"); 
             bufferFileWriter = new BufferedWriter(new FileWriter(file1,true));
             bufferFileWriter.write(oldCurrentLine);
                bufferFileWriter.newLine();
                bufferFileWriter.flush();
                resultHash.put("isexist",true);
                resultHash.put("currentRow",oldCurrentLine);
         }
            return resultHash;
   } catch (IOException e) {
        /*System.out.println("Exception-----------------"+e);
        e.printStackTrace();*/
    //   File file2 = new File("D:\\groupBy.txt"); 
  //     bufferFileWriter = new BufferedWriter(new FileWriter(file2,true));
        return resultHash;
    } finally {
        closeConnections(br,bufferFileWriter);
    }  
}
public  void temFileCreate(BufferedReader br,String currentLine,int count,Vector<Integer> vec,String delimeter)throws IOException
{
    BufferedWriter  bufferFileWritertemp1=null;
    bufferFileWritertemp1 = new BufferedWriter(new FileWriter("D:\\groupBytemp.txt"));
    String[] arry=currentLine.split(delimeter);
    String currenline1="";
    String[] oldarry=null;
    while((currenline1=br.readLine())!=null)
    {
        oldarry=currenline1.split(delimeter);
        boolean groupBycolumnsCheckFlag = checkGroupByColumns(vec,arry,oldarry);
        //if((arry[0].equals(oldarry[0]))&&(arry[1].equals(oldarry[1]))&&(arry[2].equals(oldarry[2]))&&(arry[3].equals(oldarry[3]))&&(arry[4].equals(oldarry[4]))&&(count==Integer.parseInt(oldarry[oldarry.length-1])))
        if(groupBycolumnsCheckFlag&&(count==Integer.parseInt(oldarry[oldarry.length-1])))
        {
            bufferFileWritertemp1.write(currenline1);
            //System.out.println("tempfile line :"+currenline1);
            bufferFileWritertemp1.write("\n");
        }
        //else if(!((arry[0].equals(oldarry[0]))&&(arry[1].equals(oldarry[1]))&&(arry[2].equals(oldarry[2]))&&(arry[3].equals(oldarry[3]))&&(arry[4].equals(oldarry[4]))))
        else if(!groupBycolumnsCheckFlag)
        {
            //System.out.println("temp file not equal :"+currenline1);
            bufferFileWritertemp1.write(currenline1);
            bufferFileWritertemp1.write("\n");
        }
    }   
    closeConnections(br,bufferFileWritertemp1);
    /*br = null;
    bufferFileWritertemp1 = null;*/
    File newfile1 =new File("D:\\groupBy.txt");
    if(newfile1.delete()) {
        //System.out.println(newfile1.getName() + " is deleted!");
    } else {
        //System.out.println(newfile1.getName() +"Delete operation is failed.");
    }
    File oldfile =new File("D:\\groupBytemp.txt");
    File newfile =new File("D:\\groupBy.txt");
    if(oldfile.renameTo(newfile)){
        System.out.println("success -->");
        //System.gc();
    }
    else { 
        File newfile2 =new File("D:\\groupBy.txt");
        if(oldfile.renameTo(newfile2)){
            System.out.println("???????????????success -->");
        }
    }
}
public void closeConnections(BufferedReader br,BufferedWriter bw)
{
    try {
        if(br!=null)
        {
            try {
            br.close();
            br = null;
            } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
            }
        }
        if(bw!=null)
        {
            bw.close();
            bw = null;
        }

        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
}
public String UpdateSameRowWithCount(Vector<Integer> vec,String[] oldarry,String[] arry) {
    String updatedString = "";
    for(int index = 0;index<vec.size();index++) {
        updatedString += oldarry[vec.get(index)]+",";       
    }
    if(arry!=null && arry.length>0)
        updatedString += arry[(arry.length-1)]; 
    return updatedString;
}
public boolean checkGroupByColumns(Vector<Integer> vec,String[] currentRowArray,String[] checkRowArray) {
    boolean checkFlag = true;
    for(int index = 0;index<vec.size();index++) {
        if(!currentRowArray[index].toString().trim().equals(checkRowArray[index].toString().trim())){
            checkFlag = false;
            return checkFlag; // when index is not matched
        }       
    }
    return checkFlag;   
}
public static void main(String[] args) {
        Groupingclass groupby = new Groupingclass();
        Vector<Integer> vec = new Vector<Integer>();
        vec.add(0);
        vec.add(1);
        vec.add(2);
        vec.add(3);
        vec.add(4);
        String delimeter=",";
        System.out.println("call prepareGroupBy method");
        groupby.prepareGroupBy(vec,delimeter);
    }
}

Advertisement
View Answers

Related Tutorials/Questions & Answers:
is there any possibelities fast read and write file large data file
is there any possibelities fast read and write file large data file  ...) { //read from start original file String arry[] = sCurrentLine.split...{ String sCurrentLine; // Groped file name (Suresh.txt
is there any possibelities fast read and write file large data file
is there any possibelities fast read and write file large data file  ...) { //read from start original file String arry[] = sCurrentLine.split...{ String sCurrentLine; // Groped file name (Suresh.txt
Advertisements
is there any possibelities fast read and write file large data file
is there any possibelities fast read and write file large data file  ...) { //read from start original file String arry[] = sCurrentLine.split...{ String sCurrentLine; // Groped file name (Suresh.txt
Read and write file
Read and write file  HI, How to read and write file from Java program? Thanks   Hi, See the following tutorials: Java Write To File Read File Thanks
read and write a file using javascript
read and write a file using javascript  How to read and write a file using javascript
Write data in doc file
Write data in doc file  How to write data or string in word file
Java file read write operation
Java file read write operation  how to read and write the data from text file.Suppose i have text file with 4 fields name ,roll no ,marks1,marks2 with more than 20 records......i need to store these value in object and pass
How to read a large text file line by line in java?
How to read a large text file line by line in java?  I have been assigned a work to read big text file and extract the data and save into database... you kind advice and let's know how to read a large text file line by line in java
how to read and write an xml file using java
how to read and write an xml file using java  Hi Can anyone help me how to read and write an xml file which has CData using java
To read & write a excel file using the core java
To read & write a excel file using the core java  Hai, I'm new to JavaProgram.But now i need java program to read & write a excel file so, can anyone help me to learn the above mentioned topic(link for the portion
write a progam for bubble sort using file read nd write?
write a progam for bubble sort using file read nd write?  hi, please give the code
write a program in java to read a text file and write the output to an excel file using filereader and filewriter?
write a program in java to read a text file and write the output to an excel file using filereader and filewriter?  write a program in java to read a text file and write the output to an excel file using filereader and filewriter
read text file and store the data in mysql - JDBC
read text file and store the data in mysql  when we store the data in mysql table from text file its store the data from new line to new column. how to store the data in different column from a single line of text file
splitting large xml file. - XML
splitting large xml file.  Hi, I have a large xml file(1G) and a schema (XSD) i need to split this xml to small xml files (~20M) that will be valid by the schema
read a file
read a file  read a file byte by byte   import java.io.File... static void main(String[] args) { File file = new File("D://Try.txt"); try { FileInputStream fin = new FileInputStream(file); byte
file read
file read  hi i am reaing from a file which has punjabi words. can some one help with me some code
How to read the data in text file seperated by by ',' in java using IO Operations
How to read the data in text file seperated by by ',' in java using IO Operations  in Text file data like raju 45,56,67 ramu 46,65,78 raji 34,23,56 this is the student marks in text file.this data read and calculate
simple code to write an read and write the login detail to a xml file using javascript ( username and password )
simple code to write an read and write the login detail to a xml file using javascript ( username and password )  pls can nyone give me a code to write and read the login details (username and password )into a xml file using
Large File reading through Axis2 Web service
have to write a web service which would read the XML or JSON(format) data which...Large File reading through Axis2 Web service  This is Vinay Rai.... Any suggestion/help in this would be greatly appreciated. Thanks in advance
how read data from doc file in same formate in jsp
how read data from doc file in same formate in jsp  how we can read and display data on jsp page, from doc file with the same formatting
Java write to file
Java write to file  How to write to a file in Java? Is there any... files. You can easily use the the FileWriter and BufferedWriter to write data to a text file. Here is the examples: Example program to write to file. File
Write to a file
Write to a file       As we have read about  the BufferedOutputStream class that store the data in an internal buffer and lets you write
Read specific column data from text file in java
Read specific column data from text file in java  My question is if my text file contain 15 columns and i want read specific column data from that text file then what code i should do
Read data from excel file and update database using jsp
Read data from excel file and update database using jsp  read data from excel file and update database using jsp Hi, I am using a MySQL database for scientific research analysis. My team members sends research data in excel file
How to read and display data from a .properties file from a jsp page
How to read and display data from a .properties file from a jsp page  I have a .properties file. I have to create a jsp page such that it reads the data from this .properties file and display it in table format. Ex:by using
How to upload a large excel file - JSP-Servlet
How to upload a large excel file   Dear sir , How to upload a large excel file and how to read a that large excel file or how to get a each column values from that large excel file.For small file i am getting values
How to Retrieve Data from the database and write into excel file using Java
How to Retrieve Data from the database and write into excel file using Java  Hi, I am trying to develop an small application where i trying to retrieve Data from the database and store the details in excel file. Please can
How to Split a large XML file using java?
How to Split a large XML file using java?  How can we split a 500MB Xml file?I know how to split xml file after reading the entire document in a file.Here we cannot load the entire file as it is a large file
write data to a pdf file when i run jsp page
write data to a pdf file when i run jsp page  Hi, <%@page import...{ document.close();} %> </body> I added itextpdf jar file to the libraries.the pdf file are not opened when i execute the program.please send the code
write data to a pdf file when i run jsp page
write data to a pdf file when i run jsp page  Hi, <%@page import...{ document.close();} %> </body> I added itextpdf jar file to the libraries.the pdf file are not opened when i execute the program.please send the code
write data to a pdf file when i run jsp page
write data to a pdf file when i run jsp page  Hi, <%@page import...{ document.close();} %> </body> I added itextpdf jar file to the libraries.the pdf file are not opened when i execute the program.please send the code
write data to a pdf file when i run jsp page
write data to a pdf file when i run jsp page  Hi, <%@page import...{ document.close();} %> </body> I added itextpdf jar file to the libraries.the pdf file are not opened when i execute the program.please send the code
Retrieve data from the database and write into ppt file
Java Retrieve data from the database & write into ppt file In this section, we are going to retrieve data from the database and write into the .ppt file. For this purpose, we have used Jakarta POI api to create a .ppt file. The class
Sort file data and write into another file
Sort file data and write into another file By sorting, you can arrange the data into meaningful order and analyze it more effectively. Here we have one file... and grade. The data is not properly arranged in the file. Now, we have to display
Read file in java
Read file in java  Hi, How to Read file in java? Thanks   Hi, Read complete tutorial with example at Read file in java. Thanks
java read file
java read file  Hello i need some help... i want to read an MS Excel file in java so how to read that file
Retrieve Data from the database and write into excel file
Retrieve Data from the database and write into excel file. In this section, we are going to retrieve data from the database and write into the excel file...); fileOut.close(); System.out.println("Data is saved in excel file
Java Write to properties file
. Here we are going to create a .properties file and write data...Java Write to properties file In this section, you will learn how to write data to properties file.  The properties file is basically used to store
How to write in File in Java
How to write in File in Java  Hi, How to write in File in Java..., For writing some data in the file we require some class file, objects... the (BufferedReader) constructor. So that we write some data suing
C file write example
C file write example       This section demonstrates you to write the data into the file. Here we are using the library function fwrite() to write the data into the file. You
Read from file java
Read from file java  How to Read from file java? What is the best method for a text file having a size of 10GB. Since i have to process the file one line at a time so tell me the very best method. Thank you
Read external file in PHP
Read external file in PHP  How to read external files in PHP in particular time duration and save that file with the latest date. How is it possible? Please explain with an example. Thanks
How to write in File in Java
How to write in File in Java  Hi, How to write in File in Java. Please suggest and give example of this program. thanks
Java read binary file
Java read binary file  I want Java read binary file example code that is easy to read and learn. Thanks   Hi, Please see the code at Reading binary file into byte array in Java. Thanks   Hi, There is many
Read Video File
Read Video File  how to read a video file, after that i want to encrypt and decrypt it. please help me and if u can send me some hint or source code on swarit.it@gmail.com Thanks & Regards Swarit Agarwal
Read the file Contents
Read the file Contents  Ravi Raj,Vijay45Shankar,234 Guna,345,Meet me,654,Cow Read file contents and Print the no.of words and Numbers./p> Thanks, Dinesh Ram   The given code read the file content and print
How To Read File In Java
How To Read File In Java In this section we will discuss about about how data of a file can be read in Java. A file can contain data as bytes, characters, binary data etc. To read a file in Java we can use following of the classes
How to Read a File in Java
// read via File Reader String data =""; // Read br and store a line...How to Read a File in Java? In this section we are going to know, How to read a file in Java. We have to follow three step to read a File. First get
How to Write to a File in Java
In this Java tutorial, we will demonstrate how to write to a file in Java... will be able to use OutputStreamWriter on a FileOutputStream classes to write data to a file from Java program. To write the Java Write To File we will use two
Java read lines from file
Java read lines from file  Any code example related to Java read... of reading file line by line in Java. Can any one share me the code for reading... to read the whole file in one go. So, in my case reading file line by line is only
 

 

 

DMCA.com