Ziglyph: Unicode processing with Zig

This is my very first Zig project so there’s still lots to improve all over the place, but as for now, all tests are passing and execution speed seems up to par with other languages. This is not meant to be a replacement for the Zig std.unicode library functionality; in fact, it depends on it. No pride in ownership guys and gals, so constructive criticism, suggestions, recommendations, PRs and issues are welcome of course. Still lots to learn!

8 Likes

Nice! Do you have any interest in taking this to the next level and add support for higher level operations?

Things like normalization and cleanup of invisible characters and confusables, iterating over grapheme clusters, obtaining their width, high level operations on emojis, etc?

Here’s a blog post that mentions some of these properties:

3 Likes

Most definitely would like to keep evolving in those directions. I think that achieving a solid, feature-rich, and well-performing base for text processing is a desirable goal. Dealing with Unicode, will not make it easy, but let’s give it a shot! Thanks for the excellent article link. Enjoyed it alot.

2 Likes

Awesome work @dude_the_builder ! That’s a great looking repo and I like the code examples in the README. I have no doubt I’ll need some of this functionality in a project someday.

2 Likes

Really great work @dude_the_builder ! I was just asking around on Discord the other day to see if anyone had implemented a library like this yet. I will definitely be using this in the near future for some regex engine/parser generator things :smiley:

1 Like

Just starred the repo. Definitely looks useful and worthy of continued development.

1 Like

Thanks for the support! My head is spinning here immersed in the Unicode spec! lol But it’s interesting how it offers so many areas that can be developed in different ways.