file.getEndPos() cannot use for alloc filesize?

I am struggling to do an allocation to read a file whose size is unknown at compile time. The code at bottom yields the following error at compile time:

./question.zig:37:43: error: expected type ‘usize’, found 'std.fs.file.GetSeekPosError!u64’
** return file.readToEndAlloc(allocator, filesize) catch |err| {**

Fundamentally, I do not understand why the returned type from file.getEndPos is not already a usize?

Now the code:

const std = @import(“std”);
const process = std.process;
const Allocator = std.mem.Allocator;

pub fn main() anyerror!u8 {
const stdout = std.io.getStdOut().writer();

var arena = std.heap.ArenaAllocator.init(std.heap.page_allocator);
defer arena.deinit();
const alloc = &arena.allocator;

var input: []const u8 = readFile("long.txt", alloc);
try stdout.print("{s}\n", .{input});

defer alloc.free(input);

return 0;

}

fn readFile(fileName: []const u8, allocator: *Allocator) []const u8 {
const stdout = std.io.getStdOut().writer();
const file = std.fs.cwd().openFile(
fileName,
.{ .read = true },
) catch |err| {
std.log.err(“Could not open file “{s}”, error: {any}.\n”, .{ fileName, err });
process.exit(74);
};

const filesize = file.getEndPos();

stdout.print("file size: {d}\n", .{filesize}) catch |err| {
    std.log.err("Could print filesize \"{s}\", error: {any}.\n", .{ fileName, err });
};
defer file.close();

return file.readToEndAlloc(allocator, filesize) catch |err| {
    std.log.err("Could not read file \"{s}\", error: {any}.\n", .{ fileName, err });
    process.exit(74);
};

}

Note that std.fs.file.GetSeekPosError!u64 is an error union – the call can fail, for example if you’re trying to seek through an unseekable stream (the errors are listed at file.getEndPos()). If you want to access the u64 you need to unwrap the error which is potentially returned, with either a try or a catch expression.

See the documentation on errors.

jmc,

Thanks kindly, much appreciated. Yes, that makes sense about the error … but I’m not sure how to do this the correct way.

Perhaps, a simpler way to phrase my question is:

“What IS the proper way to get a file’s size into a variable, so that it can be used to do an allocation?”

Thanks for your reply, and also for any further suggestions.

AB

My previous comment has the answer – it’s in the docs! :slight_smile:

To unwrap the error and forward it to the caller if there’s an error:

const filesize = try file.getEndPos();

or maybe handle it there somehow:

const filesize = file.getEndPos() catch |err| {
    std.log.err("this thing failed with error {}", .{err});
    return err;
};

jmc,

Thanks again for your help with this. I must do some more background reading here about error unions and try/catch.

The documentation on the fs.file.GetSeekPosError says “The Operating System returned an undocumented error code. This error in in theory not possible”.

There is a good chance that I am doing something very, very wrong and misguided. I should double check a few things and try again tomorrow after doing some more doc reading.

Thanks again,
AB

Note that readToEndAlloc's second parameter (third if you count self) doesn’t have to be the file size or the bytes to be read, it’s the maximum number of bytes you can tolerate being read. So for example, if you know your program should never handle files larger than 100 MiB, you could call it like so:

const bytes = try file.readToEndAlloc(allocator, 100 * 1024 * 1024);

So in reality, you don’t need to know the file size for this use case. If you want to handle any size the machine can handle:

const bytes = try file.readToEndAlloc(allocator, std.math.maxInt(usize));

Great, that works! A quick question on this: would this actually do an allocation of the whole 100 MB and hold it until "free"? Or, is the allocation just large enough to hold the actual filesize?

Thanks again to both dude_the_builder and jmc. I am just getting started with zig. I definitely like the language. As zig is young with scattered documentation, hints received in this forum are incredibly valuable.

I haven’t looked at the source code directly but I’m pretty sure it allocates only what it reads, so unless there’s some error while reading, this should equal the file size.

As you wrote, Zig is still young, and yet there are so many amazing things already being done with the language. I hope you stick around, this will only get better! :smiley:

readAllAlloc() uses a std.ArrayList(u8) under the hood, which grows as needed until the file is read in its entirety. By default it allocates 4 KiB chunks.

The standard library reads pretty easily, so I recommend peeking behind the curtains whenever a question remains unanswered:

Thanks for the pointer to the source code and the readAllAlloc. I have implemented this and it works nicely as well.

All the help I have received from you and dude_the_builder with this is kindly appreciated!

I now have cobbled together the ability to read in and parse json files containing vectors of data, and then plot them through a gnuplot child process all within zig. (I need to polish the code a bit … but still, it feels like progress goes quickly in the right direction in a only a short time.)

Cheers!
AB

2 Likes