Reading a File Byte by Byte in Java
Reading files in Coffee is the cause for a lot of defoliation. At that place are multiple ways of accomplishing the same task and it's oft not clear which file reading method is best to apply. Something that'due south quick and dingy for a small example file might non be the best method to use when you need to read a very large file. Something that worked in an earlier Coffee version, might non be the preferred method anymore.
This commodity aims to exist the definitive guide for reading files in Java seven, 8 and 9. I'1000 going to cover all the ways y'all can read files in Java. Too often, yous'll read an commodity that tells you one fashion to read a file, just to notice after there are other ways to do that. I'm actually going to cover 15 different ways to read a file in Java. I'm going to cover reading files in multiple ways with the cadre Java libraries as well equally two third political party libraries.
Just that's not all – what good is knowing how to exercise something in multiple means if you don't know which way is best for your situation?
I also put each of these methods to a real performance exam and document the results. That manner, you will have some difficult data to know the operation metrics of each method.
Methodology
JDK Versions
Coffee lawmaking samples don't live in isolation, specially when information technology comes to Java I/O, every bit the API keeps evolving. All code for this commodity has been tested on:
- Java SE 7 (jdk1.seven.0_80)
- Coffee SE 8 (jdk1.viii.0_162)
- Java SE nine (jdk-9.0.iv)
When in that location is an incompatibility, it volition be stated in that section. Otherwise, the code works unaltered for different Java versions. The chief incompatibility is the utilize of lambda expressions which was introduced in Java viii.
Java File Reading Libraries
There are multiple means of reading from files in Coffee. This article aims to be a comprehensive collection of all the dissimilar methods. I will cover:
- java.io.FileReader.read()
- coffee.io.BufferedReader.readLine()
- java.io.FileInputStream.read()
- java.io.BufferedInputStream.read()
- java.nio.file.Files.readAllBytes()
- java.nio.file.Files.readAllLines()
- java.nio.file.Files.lines()
- coffee.util.Scanner.nextLine()
- org.apache.eatables.io.FileUtils.readLines() – Apache Commons
- com.google.common.io.Files.readLines() – Google Guava
Endmost File Resources
Prior to JDK7, when opening a file in Java, all file resources would need to be manually closed using a try-catch-finally block. JDK7 introduced the try-with-resources statement, which simplifies the process of endmost streams. You no longer demand to write explicit lawmaking to close streams considering the JVM volition automatically close the stream for yous, whether an exception occurred or non. All examples used in this commodity use the try-with-resources statement for importing, loading, parsing and endmost files.
File Location
All examples will read examination files from C:\temp.
Encoding
Grapheme encoding is not explicitly saved with text files so Java makes assumptions about the encoding when reading files. Usually, the assumption is correct but sometimes you want to be explicit when instructing your programs to read from files. When encoding isn't correct, you'll meet funny characters appear when reading files.
All examples for reading text files employ 2 encoding variations:
Default system encoding where no encoding is specified and explicitly setting the encoding to UTF-8.
Download Lawmaking
All code files are available from Github.
Code Quality and Code Encapsulation
There is a difference between writing code for your personal or work project and writing code to explain and teach concepts.
If I was writing this code for my own project, I would utilise proper object-oriented principles like encapsulation, abstraction, polymorphism, etc. Only I wanted to make each example stand up alone and easily understood, which meant that some of the code has been copied from i example to the next. I did this on purpose because I didn't want the reader to accept to effigy out all the encapsulation and object structures I so cleverly created. That would have away from the examples.
For the aforementioned reason, I chose NOT to write these example with a unit of measurement testing framework like JUnit or TestNG because that'southward non the purpose of this article. That would add another library for the reader to understand that has nothing to exercise with reading files in Java. That's why all the case are written inline inside the chief method, without extra methods or classes.
My main purpose is to brand the examples as easy to understand as possible and I believe that having extra unit testing and encapsulation code will not help with this. That doesn't mean that'due south how I would encourage you to write your own personal lawmaking. It's just the way I chose to write the examples in this article to make them easier to understand.
Exception Treatment
All examples declare any checked exceptions in the throwing method declaration.
The purpose of this article is to show all the different ways to read from files in Java – it's not meant to bear witness how to handle exceptions, which will be very specific to your situation.
So instead of creating unhelpful endeavour take hold of blocks that only print exception stack traces and ataxia up the code, all example will declare whatever checked exception in the calling method. This will make the code cleaner and easier to sympathize without sacrificing any functionality.
Future Updates
Every bit Coffee file reading evolves, I volition be updating this article with any required changes.
File Reading Methods
I organized the file reading methods into three groups:
- Classic I/O classes that take been part of Java since before JDK 1.7. This includes the java.io and java.util packages.
- New Java I/O classes that accept been part of Java since JDK1.7. This covers the coffee.nio.file.Files course.
- Tertiary political party I/O classes from the Apache Commons and Google Guava projects.
Classic I/O – Reading Text
1a) FileReader – Default Encoding
FileReader reads in one character at a time, without any buffering. It'southward meant for reading text files. It uses the default character encoding on your system, so I have provided examples for both the default case, also every bit specifying the encoding explicitly.
1
2
3
4
5
six
vii
8
nine
10
eleven
12
13
14
15
sixteen
17
18
19
import coffee.io.FileReader ;
import java.io.IOException ;public class ReadFile_FileReader_Read {
public static void principal( String [ ] pArgs) throws IOException {
String fileName = "c:\\temp\\sample-10KB.txt" ;try ( FileReader fileReader = new FileReader (fileName) ) {
int singleCharInt;
char singleChar;
while ( (singleCharInt = fileReader.read ( ) ) != - 1 ) {
singleChar = ( char ) singleCharInt;//display ane character at a fourth dimension
System.out.print (singleChar) ;
}
}
}
}
1b) FileReader – Explicit Encoding (InputStreamReader)
Information technology'southward actually not possible to set the encoding explicitly on a FileReader so y'all have to use the parent class, InputStreamReader and wrap it around a FileInputStream:
i
2
3
4
5
six
7
eight
9
10
eleven
12
13
14
15
sixteen
17
18
19
20
21
22
import java.io.FileInputStream ;
import java.io.IOException ;
import coffee.io.InputStreamReader ;public course ReadFile_FileReader_Read_Encoding {
public static void principal( Cord [ ] pArgs) throws IOException {
String fileName = "c:\\temp\\sample-10KB.txt" ;
FileInputStream fileInputStream = new FileInputStream (fileName) ;//specify UTF-8 encoding explicitly
endeavor ( InputStreamReader inputStreamReader =
new InputStreamReader (fileInputStream, "UTF-8" ) ) {int singleCharInt;
char singleChar;
while ( (singleCharInt = inputStreamReader.read ( ) ) != - ane ) {
singleChar = ( char ) singleCharInt;
System.out.print (singleChar) ; //display one character at a time
}
}
}
}
2a) BufferedReader – Default Encoding
BufferedReader reads an unabridged line at a time, instead of one character at a time like FileReader. It's meant for reading text files.
1
2
3
iv
v
vi
7
8
9
x
11
12
13
14
15
xvi
17
import java.io.BufferedReader ;
import coffee.io.FileReader ;
import java.io.IOException ;public form ReadFile_BufferedReader_ReadLine {
public static void chief( String [ ] args) throws IOException {
Cord fileName = "c:\\temp\\sample-10KB.txt" ;
FileReader fileReader = new FileReader (fileName) ;try ( BufferedReader bufferedReader = new BufferedReader (fileReader) ) {
String line;
while ( (line = bufferedReader.readLine ( ) ) != null ) {
Organisation.out.println (line) ;
}
}
}
}
2b) BufferedReader – Explicit Encoding
In a similar way to how we set encoding explicitly for FileReader, we need to create FileInputStream, wrap it inside InputStreamReader with an explicit encoding and pass that to BufferedReader:
i
two
3
4
v
half-dozen
7
8
9
ten
11
12
13
fourteen
15
16
17
18
19
twenty
21
22
import java.io.BufferedReader ;
import coffee.io.FileInputStream ;
import coffee.io.IOException ;
import java.io.InputStreamReader ;public class ReadFile_BufferedReader_ReadLine_Encoding {
public static void main( Cord [ ] args) throws IOException {
String fileName = "c:\\temp\\sample-10KB.txt" ;FileInputStream fileInputStream = new FileInputStream (fileName) ;
//specify UTF-8 encoding explicitly
InputStreamReader inputStreamReader = new InputStreamReader (fileInputStream, "UTF-8" ) ;try ( BufferedReader bufferedReader = new BufferedReader (inputStreamReader) ) {
Cord line;
while ( (line = bufferedReader.readLine ( ) ) != zero ) {
System.out.println (line) ;
}
}
}
}
Archetype I/O – Reading Bytes
1) FileInputStream
FileInputStream reads in one byte at a time, without any buffering. While it's meant for reading binary files such as images or audio files, it can however be used to read text file. It's like to reading with FileReader in that you're reading one graphic symbol at a time as an integer and you need to cast that int to a char to encounter the ASCII value.
Past default, it uses the default character encoding on your system, so I have provided examples for both the default case, as well as specifying the encoding explicitly.
one
2
3
4
five
half-dozen
7
8
9
x
11
12
13
fourteen
15
16
17
eighteen
19
20
21
import java.io.File ;
import java.io.FileInputStream ;
import java.io.FileNotFoundException ;
import java.io.IOException ;public class ReadFile_FileInputStream_Read {
public static void main( String [ ] pArgs) throws FileNotFoundException, IOException {
String fileName = "c:\\temp\\sample-10KB.txt" ;
File file = new File (fileName) ;try ( FileInputStream fileInputStream = new FileInputStream (file) ) {
int singleCharInt;
char singleChar;while ( (singleCharInt = fileInputStream.read ( ) ) != - 1 ) {
singleChar = ( char ) singleCharInt;
Organization.out.print (singleChar) ;
}
}
}
}
ii) BufferedInputStream
BufferedInputStream reads a set of bytes all at once into an internal byte assortment buffer. The buffer size can be set explicitly or utilize the default, which is what we'll demonstrate in our instance. The default buffer size appears to exist 8KB but I take non explicitly verified this. All performance tests used the default buffer size so it will automatically re-size the buffer when it needs to.
i
2
three
iv
5
6
vii
8
9
x
11
12
13
14
fifteen
16
17
18
19
xx
21
22
import java.io.BufferedInputStream ;
import java.io.File ;
import java.io.FileInputStream ;
import java.io.FileNotFoundException ;
import java.io.IOException ;public class ReadFile_BufferedInputStream_Read {
public static void principal( String [ ] pArgs) throws FileNotFoundException, IOException {
String fileName = "c:\\temp\\sample-10KB.txt" ;
File file = new File (fileName) ;
FileInputStream fileInputStream = new FileInputStream (file) ;try ( BufferedInputStream bufferedInputStream = new BufferedInputStream (fileInputStream) ) {
int singleCharInt;
char singleChar;
while ( (singleCharInt = bufferedInputStream.read ( ) ) != - 1 ) {
singleChar = ( char ) singleCharInt;
System.out.print (singleChar) ;
}
}
}
}
New I/O – Reading Text
1a) Files.readAllLines() – Default Encoding
The Files class is part of the new Coffee I/O classes introduced in jdk1.7. Information technology only has static utility methods for working with files and directories.
The readAllLines() method that uses the default character encoding was introduced in jdk1.8 so this example volition not work in Java vii.
1
2
3
iv
5
6
vii
8
9
10
11
12
13
14
15
16
17
import coffee.io.File ;
import coffee.io.IOException ;
import java.nio.file.Files ;
import coffee.util.List ;public class ReadFile_Files_ReadAllLines {
public static void chief( String [ ] pArgs) throws IOException {
Cord fileName = "c:\\temp\\sample-10KB.txt" ;
File file = new File (fileName) ;List fileLinesList = Files.readAllLines (file.toPath ( ) ) ;
for ( Cord line : fileLinesList) {
System.out.println (line) ;
}
}
}
1b) Files.readAllLines() – Explicit Encoding
i
2
3
4
5
vi
7
8
9
10
11
12
thirteen
fourteen
fifteen
xvi
17
eighteen
nineteen
import java.io.File ;
import java.io.IOException ;
import java.nio.charset.StandardCharsets ;
import java.nio.file.Files ;
import java.util.Listing ;public class ReadFile_Files_ReadAllLines_Encoding {
public static void main( String [ ] pArgs) throws IOException {
String fileName = "c:\\temp\\sample-10KB.txt" ;
File file = new File (fileName) ;//use UTF-8 encoding
List fileLinesList = Files.readAllLines (file.toPath ( ), StandardCharsets.UTF_8 ) ;for ( Cord line : fileLinesList) {
System.out.println (line) ;
}
}
}
2a) Files.lines() – Default Encoding
This code was tested to work in Coffee 8 and ix. Java vii didn't run because of the lack of support for lambda expressions.
ane
ii
3
4
5
half-dozen
seven
8
9
ten
11
12
13
14
xv
16
17
import coffee.io.File ;
import java.io.IOException ;
import java.nio.file.Files ;
import coffee.util.stream.Stream ;public form ReadFile_Files_Lines {
public static void master( String [ ] pArgs) throws IOException {
Cord fileName = "c:\\temp\\sample-10KB.txt" ;
File file = new File (fileName) ;try (Stream linesStream = Files.lines (file.toPath ( ) ) ) {
linesStream.forEach (line -> {
Organisation.out.println (line) ;
} ) ;
}
}
}
2b) Files.lines() – Explicit Encoding
Only like in the previous example, this code was tested and works in Java 8 and 9 merely not in Coffee vii.
one
ii
3
four
5
vi
7
8
ix
10
11
12
13
14
fifteen
16
17
18
import java.io.File ;
import coffee.io.IOException ;
import java.nio.charset.StandardCharsets ;
import java.nio.file.Files ;
import java.util.stream.Stream ;public class ReadFile_Files_Lines_Encoding {
public static void main( String [ ] pArgs) throws IOException {
Cord fileName = "c:\\temp\\sample-10KB.txt" ;
File file = new File (fileName) ;try (Stream linesStream = Files.lines (file.toPath ( ), StandardCharsets.UTF_8 ) ) {
linesStream.forEach (line -> {
Organisation.out.println (line) ;
} ) ;
}
}
}
3a) Scanner – Default Encoding
The Scanner form was introduced in jdk1.vii and can be used to read from files or from the console (user input).
1
ii
3
four
five
six
seven
8
9
10
eleven
12
xiii
fourteen
15
xvi
17
18
19
import java.io.File ;
import java.io.FileNotFoundException ;
import coffee.util.Scanner ;public course ReadFile_Scanner_NextLine {
public static void master( String [ ] pArgs) throws FileNotFoundException {
Cord fileName = "c:\\temp\\sample-10KB.txt" ;
File file = new File (fileName) ;try (Scanner scanner = new Scanner(file) ) {
String line;
boolean hasNextLine = imitation ;
while (hasNextLine = scanner.hasNextLine ( ) ) {
line = scanner.nextLine ( ) ;
System.out.println (line) ;
}
}
}
}
3b) Scanner – Explicit Encoding
1
2
3
4
5
6
seven
8
9
10
11
12
13
14
15
16
17
18
19
20
import java.io.File ;
import java.io.FileNotFoundException ;
import java.util.Scanner ;public form ReadFile_Scanner_NextLine_Encoding {
public static void chief( String [ ] pArgs) throws FileNotFoundException {
String fileName = "c:\\temp\\sample-10KB.txt" ;
File file = new File (fileName) ;//use UTF-viii encoding
try (Scanner scanner = new Scanner(file, "UTF-8" ) ) {
String line;
boolean hasNextLine = false ;
while (hasNextLine = scanner.hasNextLine ( ) ) {
line = scanner.nextLine ( ) ;
Arrangement.out.println (line) ;
}
}
}
}
New I/O – Reading Bytes
Files.readAllBytes()
Even though the documentation for this method states that "it is not intended for reading in big files" I found this to be the accented best performing file reading method, even on files as big every bit 1GB.
1
2
3
four
v
six
7
8
9
10
11
12
xiii
xiv
xv
sixteen
17
import coffee.io.File ;
import java.io.IOException ;
import java.nio.file.Files ;public class ReadFile_Files_ReadAllBytes {
public static void master( String [ ] pArgs) throws IOException {
String fileName = "c:\\temp\\sample-10KB.txt" ;
File file = new File (fileName) ;byte [ ] fileBytes = Files.readAllBytes (file.toPath ( ) ) ;
char singleChar;
for ( byte b : fileBytes) {
singleChar = ( char ) b;
System.out.print (singleChar) ;
}
}
}
3rd Party I/O – Reading Text
Commons – FileUtils.readLines()
Apache Commons IO is an open source Java library that comes with utility classes for reading and writing text and binary files. I listed information technology in this article because information technology can exist used instead of the built in Java libraries. The class we're using is FileUtils.
For this article, version 2.6 was used which is compatible with JDK i.vii+
Note that you need to explicitly specify the encoding and that method for using the default encoding has been deprecated.
ane
ii
3
4
v
6
vii
8
ix
10
11
12
13
14
15
16
17
18
import java.io.File ;
import java.io.IOException ;
import coffee.util.Listing ;import org.apache.commons.io.FileUtils ;
public class ReadFile_Commons_FileUtils_ReadLines {
public static void primary( Cord [ ] pArgs) throws IOException {
String fileName = "c:\\temp\\sample-10KB.txt" ;
File file = new File (fileName) ;Listing fileLinesList = FileUtils.readLines (file, "UTF-8" ) ;
for ( String line : fileLinesList) {
System.out.println (line) ;
}
}
}
Guava – Files.readLines()
Google Guava is an open up source library that comes with utility classes for mutual tasks similar collections treatment, enshroud direction, IO operations, string processing.
I listed it in this commodity considering it can exist used instead of the built in Java libraries and I wanted to compare its performance with the Java congenital in libraries.
For this article, version 23.0 was used.
I'm not going to examine all the unlike ways to read files with Guava, since this article is not meant for that. For a more than detailed wait at all the different ways to read and write files with Guava, have a await at Baeldung'south in depth article.
When reading a file, Guava requires that the character encoding be set explicitly, merely like Apache Commons.
Compatibility note: This code was tested successfully on Java viii and ix. I couldn't become it to work on Java 7 and kept getting "Unsupported major.small version 52.0" error. Guava has a divide API medico for Coffee vii which uses a slightly dissimilar version of the Files.readLine() method. I thought I could become it to work but I kept getting that error.
1
2
three
iv
5
6
7
8
9
ten
11
12
13
14
xv
sixteen
17
18
19
import java.io.File ;
import java.io.IOException ;
import java.util.List ;import com.google.mutual.base.Charsets ;
import com.google.common.io.Files ;public form ReadFile_Guava_Files_ReadLines {
public static void main( String [ ] args) throws IOException {
String fileName = "c:\\temp\\sample-10KB.txt" ;
File file = new File (fileName) ;List fileLinesList = Files.readLines (file, Charsets.UTF_8 ) ;
for ( String line : fileLinesList) {
Organization.out.println (line) ;
}
}
}
Operation Testing
Since there are so many ways to read from a file in Java, a natural question is "What file reading method is the best for my situation?" So I decided to examination each of these methods against each other using sample information files of different sizes and timing the results.
Each code sample from this article displays the contents of the file to a string and and so to the console (Organization.out). Even so, during the operation tests the System.out line was commented out since information technology would seriously slow downward the performance of each method.
Each performance examination measures the time it takes to read in the file – line by line, character by character, or byte by byte without displaying anything to the console. I ran each test 5-ten times and took the boilerplate so every bit not to permit any outliers influence each test. I also ran the default encoding version of each file reading method – i.due east. I didn't specify the encoding explicitly.
Dev Setup
The dev environment used for these tests:
- Intel Cadre i7-3615 QM @2.3 GHz, 8GB RAM
- Windows 8 x64
- Eclipse IDE for Java Developers, Oxygen.2 Release (four.7.2)
- Java SE 9 (jdk-9.0.4)
Data Files
GitHub doesn't allow pushing files larger than 100 MB, and then I couldn't find a practical way to store my big test files to allow others to replicate my tests. So instead of storing them, I'grand providing the tools I used to generate them then you can create test files that are similar in size to mine. Obviously they won't be the same, merely you'll generate files that are like in size every bit I used in my operation tests.
Random String Generator was used to generate sample text and and then I simply copy-pasted to create larger versions of the file. When the file started getting also large to manage within a text editor, I had to use the command line to merge multiple text files into a larger text file:
re-create *.txt sample-1GB.txt
I created the post-obit 7 data file sizes to test each file reading method beyond a range of file sizes:
- 1KB
- 10KB
- 100KB
- 1MB
- 10MB
- 100MB
- 1GB
Operation Summary
There were some surprises and some expected results from the performance tests.
As expected, the worst performers were the methods that read in a file character by character or byte by byte. Merely what surprised me was that the native Java IO libraries outperformed both 3rd political party libraries – Apache Commons IO and Google Guava.
What's more – both Google Guava and Apache Commons IO threw a java.lang.OutOfMemoryError when trying to read in the i GB test file. This also happened with the Files.readAllLines(Path) method but the remaining seven methods were able to read in all test files, including the 1GB exam file.
The following tabular array summarizes the average fourth dimension (in milliseconds) each file reading method took to consummate. I highlighted the elevation three methods in green, the average performing methods in yellowish and the worst performing methods in red:
The post-obit chart summarizes the above table but with the following changes:
I removed java.io.FileInputStream.read() from the chart because its performance was so bad it would skew the entire chart and you wouldn't see the other lines properly
I summarized the information from 1KB to 1MB because later that, the nautical chart would get too skewed with then many under performers and also some methods threw a java.lang.OutOfMemoryError at 1GB
The Winners
The new Java I/O libraries (java.nio) had the best overall winner (coffee.nio.Files.readAllBytes()) simply it was followed closely backside by BufferedReader.readLine() which was also a proven meridian performer across the board. The other excellent performer was coffee.nio.Files.lines(Path) which had slightly worse numbers for smaller exam files but really excelled with the larger test files.
The accented fastest file reader across all data tests was java.nio.Files.readAllBytes(Path). It was consistently the fastest and even reading a 1GB file only took well-nigh 1 2nd.
The post-obit chart compares functioning for a 100KB test file:
You can meet that the everyman times were for Files.readAllBytes(), BufferedInputStream.read() and BufferedReader.readLine().
The following nautical chart compares performance for reading a 10MB file. I didn't bother including the bar for FileInputStream.Read() because the performance was then bad it would skew the unabridged chart and you couldn't tell how the other methods performed relative to each other:
Files.readAllBytes() really outperforms all other methods and BufferedReader.readLine() is a afar 2d.
The Losers
Equally expected, the absolute worst performer was java.io.FileInputStream.read() which was orders of magnitude slower than its rivals for most tests. FileReader.read() was also a poor performer for the same reason – reading files byte by byte (or graphic symbol by character) instead of with buffers drastically degrades performance.
Both the Apache Commons IO FileUtils.readLines() and Guava Files.readLines() crashed with an OutOfMemoryError when trying to read the 1GB examination file and they were near average in performance for the remaining test files.
java.nio.Files.readAllLines() too crashed when trying to read the 1GB test file merely information technology performed quite well for smaller file sizes.
Performance Rankings
Here'south a ranked list of how well each file reading method did, in terms of speed and handling of large files, equally well as compatibility with different Java versions.
| Rank | File Reading Method |
|---|---|
| 1 | java.nio.file.Files.readAllBytes() |
| 2 | java.io.BufferedFileReader.readLine() |
| 3 | java.nio.file.Files.lines() |
| iv | coffee.io.BufferedInputStream.read() |
| 5 | java.util.Scanner.nextLine() |
| half-dozen | java.nio.file.Files.readAllLines() |
| vii | org.apache.eatables.io.FileUtils.readLines() |
| eight | com.google.common.io.Files.readLines() |
| 9 | coffee.io.FileReader.read() |
| 10 | java.io.FileInputStream.Read() |
Decision
I tried to present a comprehensive set of methods for reading files in Coffee, both text and binary. We looked at xv dissimilar means of reading files in Java and nosotros ran functioning tests to see which methods are the fastest.
The new Java IO library (java.nio) proved to be a great performer but and so was the classic BufferedReader.
simpkinsyousadvater.blogspot.com
Source: https://funnelgarden.com/java_read_file/
0 Response to "Reading a File Byte by Byte in Java"
Post a Comment