Summary: #

This is an interview with the president of the Veilid foundation, lead Rust developer and long term Cult of the dead cow Hacking group member Christien “DilDog” Rioux.

How does he see rust?
Where is Veilid going?
This and a lot more in this interview.

If you prefeer audio instead of reading, you can find the audio link at the end of the page.

How did you start with Rust?

I got started with Rust when I was working at Apple, I was given a lot of research time to focus on new languages and some of the developement in the secure language space. I was previously the founder of a company called Veracode and I spend a lot of time working on securing languages and had a lot of my own ideas on how to improve the guarentees a compiler can make.

So I was already kind of fresh on the idea of propagating more guarantees across translation unit boundaries and things like that, building a more whole graph of program guarantees. Veracode kind of does a lot of that internally, but that’s after the fact. That’s after you’ve built and shipped your product, you run a scanner on it and you find bugs.

What if the compiler could find whole classes of bugs that previous compilers couldn’t, right?

Nothing is quite as good as eliminating whole classes of security flaws rather than finding them and then fixing them. So after having spent the the lion’s share of my career writing detections and reverse engineering, I kind of turned my brain toward you know what is the possible ways to improve software security.

You know just through better languages and better software design. And I noticed that Rust was a thing that had been around a little while, showed a lot of promise, and was looking to solve a lot of the same problems.

After I left Veracode, I went to Apple for a while, was working on some scanning technology there. And eventually, when I kind of came out of that and ended up going to Lacework and other other stuff,

I had chosen ah Rust as the core language for Veilid, which is one of my private projects that you know is now is public big open source private application development framework.

Rust It was very attractive, not just for the security properties, but also that it had a lot more native ecosystem for cross-platform builds.

And for me, you know eliminating the complexity of rolling your own cross-platform tool chains all the time ah made it very attractive. The fact that it had already been ah an early adopter of WebAssembly made it very attractive as well. I wanted to build something that was mobile first and web first as well as like supporting native platforms. Bot having to write the core multiple times and you know share a lot of code between the Wasm and native builds made Rust very attractive.

What year was this?

I discovered Rust probably five or six years ago now. It was a bit early, but not like so early that you know they did not have the question mark operator yet or anything like that.

Russ had moved away from the try macro(The try macro in Rust is a feature that was introduced in Rust 0.12 and removed in Rust 1.39.0. It was used to propagate errors from a Result type, similar to the ? operator introduced in Rust 1.39.0) and stuff like that. And we were starting to see what became modern Rust. I think we had just done the 2018 Rust at the time.

So I was moving from 2015 to 2016 to 2018 Rust and started writing some of my first test programs in that.

Veilid quickly became the means by which I learned a lot more deep Rust. I had already been programming in almost every low level and high level language, for the last 40 years. Anyway, this was still an opportunity for me to sort of learn what design patterns I had that were actually anti-patterns.

Rust taught me a lot about how to be a better programmer just by learning it, forced me to think about concurrency differently, locking models. It was one of the first async runtimes that I you know had used you know that the whole executor model for Async was the first time that I really peeked into how Async worked on all of these languages.

I mean, you kind of got it for free with Node.js, TypeScript and whatever. But, its all abstracted away. As well as the notions of you know data atomicity. You don’t think about that stuff in some of these high-level languages. And Rust forced me to think about that stuff and how I would apply it to Veilid and in general. Ended up writing a lot of my own tools for concurrency and you know shared data.

BGecause some of the the stuff that’s in the Rust standard library and Tokio or AsyncSTD or parking lot, some of these really popular libraries that do concurrency is part of their mission. We’re missing a number of useful idioms that I found i that i that I wanted to have. So part of the Veilid Um, project was, you know, you’ve got Veilid core, which implements everything, Veilid platform API. Then there’s this like, not as famous back end toolkit library called veilid tools.

That is sort of just as a bag, a collection of arbitrary utilities that might be useful outside of Veilid. But, it’s quickly growing into very useful rust idioms and can be used by any other projects.

The Veilid tools crate. Is that stuff that’s built with the Veilid core?

No, it’s the other way around. Veilid Core uses Veilid Tools as like its you know bag of stuff crate.

However, a lot of the things that are in there could be probably be factored out into their own independent crates, things like the eventual class. That structure is a ah way of, it’s sort of like a channel, like a resetable single shot channel.

But it has some really interesting data sharing properties where basically you’re saying it’s sort of like a reverse future. You’re saying that eventually a value is going to show up in this data over here and you just can kind of check it when you want or you can wait on it. So it’s sort of like an inverse future. You can resolve it multiple times or a single time. You can resolve it with something that can clone versus not. There’s a bunch of tools that are like that. There’s another one that’s like the asynchronous table lock, which is sort of like being able to make a table of things where each individual item in the table is independently locked. So it abstracts away needing to have a whole ton of mutexes for individual pieces of data. There’s some just some interesting constructions there.

And I actually ended up taking a lot of those tools, and one of Veilid’s ah ah first-class bindings is the Dart language used by Flutter, which is a cross-platform, high-level language that is sort of like a Node.js, but it’s it’s used by primarily by the the Flutter framework to write cross-platform applications.

Dart:

https://en.wikipedia.org/wiki/Dart_(programming_language)
https://dart.dev/
https://en.wikipedia.org/wiki/Flutter_(software)

I really like Dart. I see Rust is sort of like a middle tier language, where like low level would be like assembly and mid level would be like Rust.

Dart is a good high level complement to Rust, it turns out. It has some great asynchronous properties. It’s strongly typed. It makes sense for those two languages to play well together.

So I ended up implementing a lot of the Veilid tools utilities in Dart as well, just so that there could be some commonality between code written in Dart, code written in Rust.

Anyway, I’m very excited and bullish on Rust! I like it. I want to also investigate some other languages that are nearby Rust like Zig and things like that. Those are all very interesting to me.

Links:
https://en.wikipedia.org/wiki/Zig_(programming_language)

I haven’t had any you know sort of commercial opportunity to do that in any of my jobs or my but open source projects. But Rust is here for the long term.

I’m happy with it and and Veilid is better for it.

How did you think the Rust syntax was like in the beginning? Was it easy for you to coding in Rust?

Yeah, generally speaking.There’s a few pieces of the syntax that might have not been totally intuitive. But syntax wise, it’s fine. I mean, obviously, everybody has to get over the idea that lifetimes are a thing and that there’s a syntax for them.

And what are they conceptually? That’s less of a syntax thing than it is just like a general concept. But once you start thinking of lifetimes as types in a way, you know they they kind of are like types.

They’re like a level above types in the type hierarchy from the same meta type concept as template parameters. Once you start thinking about them that way, it’s a little bit easier to understand.

There are times when obviously we all fight the borrow checker. There are times when I feel like I should be able to reason my way around mutably sharing an object and a piece of that object together.

The whole owning ref crate, which is designed to keep a mutable or a reference to a thing and a member of that thing, a field or a part of it together so you can pass them together with the same lifetime.

That whole crate is a bit of a hack and it’s kind of fallen apart and there are a lot of soundness problems with it.

Links:
https://kimundi.github.io/owning-ref-rs/owning_ref/index.html
https://users.rust-lang.org/t/whats-the-current-state-of-the-owning-ref-crate/102098

It would be great if the Rust compiler solved owning ref and some thing it was part of the standard library that did that but was sound, you know.

There’s more than one time when I’ve got like a mutable thing where I know I’m going to be possibly touching any part of it. But for the purposes of organization, I want to be able to call a function and pass in just a piece of it. I can’t really do that without and and pass it and a part of it it together without some help.

I just don’t feel safe using owning ref because there’s just so many little pitfalls. And then there’s like three or four different forks of that. And people argue about what the right way to do it is. There’s no great successor for it either.

So from a language design standpoint, that’s one thing that I kind of miss because I bump into it a lot.

Syntax wise, though, you know it feels enough like a C++ plus plus or a Golang. for me to be like, yeah, it’s a semicolonish language. And you know I understand the colon colon type format ah to scope path specifier. and The template format looks like a C++ plus plus template. So for me, it was close enough. Dart kind of feels the same way. So you know they never use colon colon.

There’s Very few places where I have to go refer to a syntax guide for Rust.

The one I always have to look up is how do I properly treat a concrete type as one of its a specific one of its traits.

Casting to a trait in the right way. is a little tricky. I mean, you shouldn’t have to do that that often, but it does happen.

And every time that I run into that one, I’m like, god, dang, which freaking syntax is that? And i I’m like, do I just use as? No, I don’t want to do that, because it’s like I want to make sure that I can you know i get a dynamic type check it and and whatever. And I just keep, is it into? And I always end up looking it up. There’s a few that are like that. But you know generally speaking, I don’t have a problem with Rust.

What is your favorite crates?

I really do like Tokio(https://tokio.rs/).

Tokio is great. I’ve learned a lot from that. I use it in Veilid.

It’s the kind of thing that just feels like it was like a natural extension of of the standard Rust library. I immediately moved to every everything.

Yeah, yeah. but All the async stuff and the network stack is it awesome.

There’s a few things that I kind of wish they had done more like the AsyncSTD project, but that one kind of looks like it’s on its way out.

I still support it. Veilid lets you pick an executor, so you can use either one internally, and it’s turned on and off with features.

And I wanted to make sure that it was Veilid is not tied to a specific executor because of also we build for the wasm32-unknown-unknown target.

Read more about wasm and Rust here: https://rustwasm.github.io/docs/wasm-pack/

Tokio has a hacked version that supports using WebAssembly, like Wasm bind gen futures or whatever, as is back end or something like that.

There’s like a Wasm Tokio unofficial crate that is close, but it’s not exactly the same. But anyway, you can’t really use Tokio for Wasm today. And we’re not completely stable on Wasm 32, the Wasm32 target. Because we’re only using Wasm in browser, it doesn’t matter anyway.

Ee wanted to make sure we could swap out the Wasm bindgen futures executor for Wasm. So we actually support three different executors today and it all works seamlessly.

And our testing pipelines test the same on all of them. It was a good choice to do it for now.

Right now, can I compile a node to WebAssembly and then run it in the browser?

There’s there’s a few different things in that question.

First off, Veilid Core it builds to a library that does all of the heavy lifting of running a node.

Now, if you’re writing a mobile application and you link in Veilid Core, your application will become a node. If you are trying to run Veilid Server, which is a very simple application that does nothing. It’s just there to help the network out. That only runs on native platforms today, Windows, Mac, and Linux.

We do have published applications running on Veilid today that run in the browser, running a Veilid node in the browser.

So if you wanted to see one, you could go to:

http://intersect.blog/

And that’s is running a Veilid node in the browser, connecting to all of the other native or Wasm nodes that are out there, and not performing any other HTTP requests to the server.

Once it gets loaded into the browser, it’s all web sockets that connect out to all of the other nodes. And that’s all in WebAssembly and in the browser. That’s the one that I know of that that is fully WebAssembly and is all Veilid in browser.

How did Veilid start? What sparked the idea for Veilid?

We saw Twitter starting to get bad. A bunch of friends of mine and I at the Dead Cult of the Cow started looking at the state of social media, bemoaning the lack of privacy in social media, and decided to try to do something about it. We ended up backing into some other technologies that might be able to help like IPFS. was one of the first ones we looked at like, okay, that’s peer to peer, it can store data, that’s a good start.

And I looked at it, and unfortunately, it was kind of too slow to really be useful. More to the point, it really was not well supported on mobile devices. And if we were going to make a social network or social applications, they needed to be something that every person out there could use. I didn’t want to make just another peer-to-peer platform for nerds. I wanted to make it so grandma and my kids and people at my office and that that aren’t technical, I want everybody able to use it.

So it had to be zero config. It had to not be like, oh, run through a proxy like Tor. Now I loved the privacy guarantees Tor was giving people. So I was like, onion routing is a good thing. We want something like that. We also want distributed storage. So something like IPFS.

So what if we tried to put something together that had those properties with the strong cryptography end to end but was also mobile first?

What would you have to do?

How would you design a system that was mobile first but had those characteristics?

Veilid came about by by noting that nothing else could built a network that survived high churn. One of the properties of mobile device-oriented networking is that it you have devices that are constantly switching between Wi-Fi access points, going from Wi-Fi to cellular to different IP addresses, and you’ve got NAT problems.

And then you’re at home and you get a port addressable, and you know yeah whatever. All a lot of different network, the reality of the network is that you know nodes are going to be coming and going a lot more. Higher churn, many different kinds of networks. IPv4 sometimes, IPv6 other times, both. um Sometimes even crazier things like VPNs like that you’re using from your device.It could it could be ah a more complicated network environment and all of that had to just work.

We didn’t want to centralize anything, because we didn’t want the responsibility of running a centralized thing. As soon as you build a privacy and-oriented network, everyone’s coming at you with what you know someone’s using your network for the bad thing, whatever it is today, we need to demand access to this user’s account.

Well, we can’t give you that because we don’t have it. Nobody’s got it but that user. You want access to their stuff, you got to go get their their phone and you know figure out how to dump it. And that’s the only reason Signal and other end-to-end encrypted apps are still around is because all they’re doing is moving bits around. you know If you want access to the encrypted stuff, you got to get the device and get the keys and and whatever else. And that’s fine. We wanted to build something that you know it could give people that end-to-end security, but also target app development models that encourage the monetization of people’s user data.

And centralization just has that problem. You’ve got an AWS bill to pay. You’ve got advertisement that might be used to pay for that so that it can look like your app is free.

All of that is putting user privacy in a place where it’s monetized to pay the bills.

There’s the wrong incentives. Could we build a framework that but allowed you to use the fact that you have a supercomputer in your pocket?

So does billion other people on this planet. That’s untapped resource right there.
It’s an opportunity for people to help each other out across the internet to build application state and like the experience of an application without needing to go through some central choke point.

It’s not to say that everything can be decentralized. Maybe not everything can. But things that can be decentralized probably should be, you know, not just for the financial aspect but also just if you don’t have users data you don’t have to be responsible for it and I think that’s important. You know, I shouldn’t have to worry about whether or not people are discussing sensitive medical history stuff across VeilidChat and worrying about if that gets hacked, is that going to go right to some insurance company that you know denies their grandkids someday? you know I don’t want to be have that on me.

I just want people to be able to chat.

We’re building the future of application communication.I hope that we can get more people on the platform helping build it. There’s a lot more to do, but I’m pretty happy with what we’ve done so far.

What went through your mind when you like started designing the stack of Veilid? Did you base the source code on Previous repos?

I kind of looked a little bit for inspiration ah from my IPFS’s use of RPC the way it was doing RPC.

To some extent, also Bitcoin, I mean, I didn’t want to build a blockchain at all into this. I specifically didn’t want an immutable ledger to be any part of this, though I respect the concept of a blockchain for a blockchain for doing that.

It didn’t make sense as a communication or a signing mechanism or anything like that. I also didn’t want people chalking it up to be yet another Web3 coin play. So many peer-to-peer things are just that, and I didn’t want to go that way.

Anyway, IPFS had some designs around DHT usage for distributed hash tables and block based sharing for content addressable hashing.

Those concepts were good. I read through their source. And you know when I say the IPFS source, there’s like three different, four different core IPFS implementations out there. So I looked at the Go one, and then I kind of looked at the Node.js one.

I was just like, all right, well, I can’t use any of this.

But I like some of the ideas. So I’m going to borrow what I can.

I’m going to build a design that is similar to Tor in terms of packet envelope encapsulation and pealing off process.

But I can’t have any machine on the network be special. I can’t have any kind of gateways, like Tor has like entry and exit nodes that are a little different than a guard node versus whatever.

These nodes have different roles. For Veilid, everything is in network. You don’t enter or exit Veilid. You just use your node and operate only on the network. So fundamentally, we had the opportunity to make everything completely homogenous. No node is special or different. And because of that, ah you know we could make some slightly different choices about how to construct routes.

For example, we don’t build the route all the way to the end. We we build half the route. And then the other half of the route comes from the destination. And then we snap them together. So you can form the safety route which is sender privacy, and then a receiver privacy private route, and they combine together cryptographically to form a a combined route. And there’s some provenance type concerns that are are mitigated by that, like like how do you minimize the risk that some node is compromised? You get to pick at least half the nodes in the route.

And the destination gets to pick the other half that way they can both mutually agree that like not the whole route was chosen by possibly some attacker so we had to think about that as far as crypto cryptographic suite and stuff like that.

I wasn’t married to any particular cryptographic concepts of other than like just we got to have a hash that works in this nice and fast. We have to have a reasonably large a key space for everything to operate in so that we minimize any kind of collision-based attacks.

So I chose a 256-bit key space just because it was bigger than a lot of the other spaces that people use. And I just acknowledged that the cryptography suite was going to have to be upgradable.

You know, I looked at TLS and some of these other things where it’s like every couple of years, something falls out of fashion.

They’re like, shouldn’t use SHA-256 anymore. It’s bad or whatever. So I just assumed that being able to support multiple crypto systems at the same time was going to be important.

So all of the Veilid node IDs and the authentication and signing that we do and all of the encryption and DH and you know key exchange and a key agreement type stuff that we do is all pluggable.

And we have one cryptosystem right now that was based on my conversations with some cryptographer friends of mine. They were like, yeah, if you choose these things, you’re not going wrong, right? So I put the VLD0 crypto system in and said, look, one day we’re going to move to a VLD one crypto system.

And I made I made another crypto system just called I call it none. It’s not like an it’s like VLD zero, but everything is just kind of dumb. And I made it, turned it off by default, but I used it to test having two different crypto systems running before I published Veilid. That way I could say like if you had like a VLD zero key and a none key, they could still talk to each other, but they’d have to use the each other’s cryptography to do it.

That way, when VLD one does show up, VLD zero nodes and VLD one nodes will be able to coexist.

And then eventually, you know, a year later, we can deprecate VLD zero and hopefully everyone’s upgraded. If not, they they start to get kicked off the network and stuff. So. Yeah, yeah, just make sure that everything’s upgradeable. Don’t make any decisions. You can’t back out of design things ah for high churn. And just assume that, you know, there’s going to be a wide range of different capabilities available to nodes. Some are going to be stable and can help out the network more. Some are going to be like on a crappy, 3G going through a tunnel and not able to do much besides, you know, read a few DHT keys and send a few village app messages. Then it goes offline.

Has there been any severe bugs found for Veilid? Or really bad bugs that got pushed and then half the network went down?

There haven’t been any ones that have kicked the network offline, but I’m sure that there’s there may be something that happens eventually.

I would expect a denial of service attack at some point to be successful, though the distributed nature of it does make it hard.

You kind of have to attack every single Veilid node to to get anywhere. Those are highly distributed around the world.

So it’s not like you can just target one part of the world, which is pretty common. you know Usually it’s focusing on one node. But no one node, we we designed this thing to expect that nodes go away all the time.

If you focus on a handful of nodes, the rest of the network might not even notice. All of the data that we store in the DHT is stored replicated all over the place. There might be ways to to target a Veilid user.

If I were to like think of a way right now, it would be figure out what nodes are close to or would store a specific DHT record that you’re interested in and denial of service to those nodes specifically.

Right now, the network might take a while to sort of catch up and that data may become inaccessible for a little while. Again, that wouldn’t affect everybody, but it would affect like the one person that cares about that record. So there might be some ways to in a very targeted way, slow down or make access to some Veilid data more challenging. But you asked a very specific question, have there any been any big attacks?

There have been security vulnerabilities, not like remotely exploitable per se, but like, signatureing issues that needed to be corrected where people could like write to DSH records that shouldn’t have been able to. And there there but there were a few vulnerabilities that I would say needed correction.

We actually we do full disclosure. So when someone reports to us, we take the exploit code and we check it into the repo .

That’s very impressive.

We’re probably the only project I know that does this, the exploit code, not just like the not just the advisory.

We take the exploit code that breaks Veilid and we put it in the repo for Veilid. This encourages everyone to upgrade.

How do you effectively communicate new releases and like spread the message that its time to upgrade?

We run our own RPM and deb repos for the Veilid server. We have packages that we build um that are blessed we sign them, we push them out you can when if you go to the Veilid GitLab, which is where our repository is. There are instructions on how to set up a node and set it up to automatically update to the latest Veilid server. if you’re just building a Veilid app, you’ll get the crates when they come from crates.io, you can just always do a cargo update and it show up.

Is it just like a extra flag you add when you start a node to like just set them forget to auto state or? to auto upgrade it

No, there’s usually an operating system specific unattended upgrade mechanism that we encourage people to turn on.

And we give people guides on how to do that. Not enough people upgrade today. And I would love if we could get more people to do it in an automated way. We’re going to be working on that.

Now, there is one little tiny piece of centralization that we do have at our disposal that can encourage upgrades. And that is the fact that we still run bootstrap nodes. for Veilid Now, in theory, you know, our bootstrap nodes are no different than anybody else’s. Other than the fact that if you don’t have any nodes in your routing table and you need to go find some, you go to our bootstrap nodes to find your first few nodes. And then from there, you can just talk to anybody. Now, if we want people to upgrade, one thing we can do is we can push a breaking change to the protocol and make sure it gets to our bootstrap nodes.

What will happen is older nodes will start to get banned by other nodes. So one of the properties that we have is this thing called punishment. Now, punishment is not ah it’s not like a banned list where you know people are sharing punishment info.

But if I have three nodes and they’re all version one, now version two comes along and two of those nodes speak it. But version one does not and version one tries to communicate with two and it can’t those two nodes are both going to look at one and say like ah don’t put us don’t you’re not going into our routing table we’re going to ignore you because you don’t speak our language anymore, so there’s various things that can happen that cause a node to get punished one of those things is like spamming or it can be punished by node id it can also be punished by IP address so if you’re like hammering too many packages trying to denial of service it’ll just lop it off and say nope that notes gone and and it’ll get banned from the network for an hour.

One thing we can do to encourage upgrading is just make a breaking change to the protocol. and the nodes that can’t talk to all those other nodes anymore will just end up siphoning off into their own little private Veilid and you know they just won’t be what everybody else talks to anymore so yeah we have ways of doing it they’re a little more aggressive um and someday i would love it if there was more like like a way to um have the the binary itself, like just download the new core every once in a while. But until we have our block store operational, I don’t see that happening. But you know also it’s a trouble it’s a trouble for mobile devices as well. like you can’t re-download a a shared library in an iOS application and run new code.

S it would have to be like a web assembly blob or something like that, where it’s actually interpreted and there’s a whole just a time compiler thing there. And, there’s some technical challenges to making the core auto update without literally replacing the binaries on disk.

I’m a developer and I want to get my hands dirty with Veilid, Where do I start?

Well, at first, you’ll probably want to do two things. One is join the Discord. I know a lot of people don’t like Discord, but it’s a great place for you to meet other Veilid developers and talk about their experience .

There aren’t a lot of pure Rust applications that build Veilid in today. The flagship app is Veilid Chat. We’re still actively developing that. But there are other people in our Discord that have Rust apps that they’ve built. That Intersect app is not open source yet. But what once it goes open source, it is written in Rust.

So you’ll be able to talk to others that have experience with that. There are Rust apps that are not finished yet.

We don’t have a full Rust sample code base yet. So talking to other people on the Discord is probably your best bet. TI think that you’ll find that the Rust docs are coming along.

We have a book as well. It’s about half done. MD book.

We have an MD book that’s also available off of our GitLab ah that explains you know the fundamental concepts, so it gets you an idea of like you know what you’re looking at.

Now, there’s another language that we support as well, and it is not as sort of tidy, but it is pretty quick to get started. And that is Python.

You can run a Veilid server and connect to the Veilid server with Python and have an application written in Python. We have a first class async Python API with type hints and everything.

So very, very modern Python.

And we have a sample chat app that works today that you can use to get an idea of how you might use the Veilid API. Just play with that, read through it, it’s easy to understand, and then kind of get you come back into ah writing your Rust app from there.

Do you have any applications in mind of like proof of concepts or like cool applications, that people could build as a weekend project using Veilid?

Yeah, yeah. on the Discord, there’s a whole forum called App Pitches and Ideas.

And a lot of people have contributed concepts here.

I’ve got at least 20 to 30 ideas that some other people would like to build themselves or would love others to try to build. Open street maps, screen sharing over Veilid, a WordPress killer, Veilid Rust on Rails, like a template for building web apps that use Rust, decentralized wikis, you know how to use Veilid with Terraform, a bridged activity pub, a Discord alternative, TikTok replacement.

I mean, there’s all these different things one could probably do.

Not everyone’s like at the same sort of level of technical capability to be able to to get it started. But as these open source source projects go, someone who’s well versed with the library and with the language can get the ball rolling and then build critical mass with developers who are trying to learn.

That’s kind of what we’re doing with Veila too. You know, it’s obviously a huge upfront startup cost to understand how to contribute to the Veila core.

But there are people doing it. I’ve got some really talented folks that are coming up on it and are are making pull requests. We accept PRs or emerge requests from forks all the time and people are adding interesting features, we’re all about having people help on the core on Veilid chat if you want to play with dart as well. And there’s a lot of ideas on the discord and people ask all kinds of interesting questions and have a lot of ideas so feel free to to pop in there and pick off a project or contribute to somebody else’s.

Join Veilid discord now: #

https://veilid.com/discord

We had about 3,500 people there. And I’d say about,you know, 50 or 60 that are regularly hanging out there.

What if I’m a non-developer or I’m not a techie, but I want to contribute to Veilid, How can I do that?

Join the discord we have an off topic and human we have basically two sides of Veilid, you got the technical side, and then you’ve got the sort of human side of like, how do you get this into people’s hands how do you get the idea to conferences and developers and like how do you proselytize this stuff. We have people always going and talking about it at events.

We have people making media, memes and building communications around it, trying to get it in the hands of people who are politically minded involved, you know legislation, trying to help people influence the politics of the world and the ways that would help, strong encrypted applications. I mean, there’s, this is a general fight for how we communicate and for the rights of people to just talk to each other and not you know go through a middleman. So there’s a huge political side to this, that you know whether we love it, to if this is just a technical problem, it’s not.

There’s a huge human element here.And if yeah if youre if you’re someone who is like like aware of the technology and and the surrounding challenges, isn’t a programmer necessarily, but gets what Veilid is doing, but you want to help bridge the gap, educating the masses, making media like tutorials and videos.

Helping the social media efforts out. All of that stuff is important. We’re not going to get you know, to be done with a Veilid until we have, you know, pulled people’s grandmas off of Facebook. I mean, this is the kind of thing that is the future we want to build.

And that doesn’t come just by, you know, having a bunch of techies talking to each other and not reaching out into the world around us. So, you know, we want to speak at developer conferences. We want to speak at, you know, political events. We want to speak at, going to get out there and make change through, making it more visible.

There’s things that people can do. You don’t have to be a coder. You just have to understand the problems and be willing to, , get out there and help people understand it and and contribute it however they can.

That’s awesome. If we look at the future, ah where do you think the Veilid project will be in five years?

Well, five years, I would love to have more than a couple apps on it. you know I’d like to have it be the core of 20 to 100 apps, maybe.

And and you know in the hands, I’d like to have 100,000 nodes instead of 500 nodes. you know I think the scale needs to happen, and I think we can get there.

These things tend to take off. They don’t all have to be you know heavy lifting, Veilid server type nodes. Every node in Veilid is contributing in some way. So if you’re running a browser like I’m talking to you with Chrome right now, if there was a Veilid plug-in there that when you ran your browser, it also ran a Veilid node, you know people might be able to just easily, with a click of a button, help the network out without needing to know it how they were helping. We could find ways to get nodes into people’s hands. Make it easier for people to contribute resources and build the network out. Just the fact that you’d be running you know possibly 20 or 30 other apps that are Veilid apps, those are all nodes. So yeah, I think that the scale is going to be much more impressive in five years if we’ve done our job right. I think we’ll be very actively getting people to use a Veilid chat as their primary chat application. I hope to get our own community off of Discord and onto a Veilid chat-based communication platform. Y I believe in dogfooding everything. I want us to use Veilid apps to talk about Veilid. I want to not use Zencaster. I want to use a Veilid-based streaming and recording solution.

Those kinds of things are the future.

I want to get us there. and And frankly, you know, all the heavy lifting right now is being done in the browser when I’m talking to you. There’s no reason why that couldn’t then back end out to, you know, a Veilid private route and eventually the block store where things are cached. There doesn’t need to be something in the middle necessarily, you know. I can see a world where it where where Veilid apps kind of start to take over.

One of the unexpected developments I saw it getting used. Somebody made a plugin to the Rust Bevy game engine that uses Veilid as the communication platform for Bevy.

Links:
https://bevyengine.org/
https://github.com/bevyengine/bevy

Instead of going through Xbox Live, you could just go through Veilid, and suddenly you’re chatting and playing with hundreds of other people, playing this one game in real time, rather than requiring some central Battle Net or Xbox Live type thing.

That to me is a big deal if you didn’t need any of those gaming services anymore.

There’s so many awesome the use cases for Veilid.

Yeah, so many. There’s a lot. I am looking forward in five years to being surprised by how people use it. I can’t predict it. I’m just trying to make room for people’s ideas. And every couple of months, something comes up and I’m like, OK, I didn’t think of that. you know So I would expect that to continue to happen.

Where are we now? What’s coming up in the next release? What are you currently working on? Is there any cool features coming up?

This is an interesting time where we’re starting to turn the corner on scale. Yes, there are some interesting features coming up. This next release is probably just going to be more of a flat release, not really new features per se, but ah it’ll be designed. There’ll be a couple, but they’re mostly internal. The goal being to kick off old nodes. I’d say that half the nodes in the network need to go because they’re not being upgraded.

We have a lot of really cool features that aren’t working very well today because old nodes are just aren’t supporting them and there’s too many of them.
We’re going to be pretty aggressive with this next release about kicking off old nodes. The release after that I’m currently working on a way to simulate an entire Veilid network.

So that apps can be written without needing to just to use the whole Veilid network onto itself. There are basically this boils down to simulating the entire internet and running, like a testnet but without actually using the internet like just having a few processes.

Let’s say you have a process that can run 20 nodes in a single process. And then those talk to each other like multiple copies of that process. Now you’ve got 100 nodes running on a single machine that’s very lightweight. So you can practically do that now. But have it simulate and think that it’s on the real internet. This will allow us to roll out large scale algorithmic developments like distributed algorithms.

Right now we have to do a whole release, get people to download it, put it on the real internet because you can’t simulate Veilid.

There’s just no good practical way to do it. The thing I’m working on right now is a whole internet simulator built solely for Veilid, that can do hundreds of nodes and test our distributed algorithms.

A bunch of other things that are coming are things like, it’s an internal detail, but being able to publish private route information automatically on our distributed hash table and have it auto refreshed and in done in parallel so that if routes die, it automatically fails over to a new route without any downtime.

So this is that that’ll be the thing that enables like end-to-end streaming between private endpoints ah without worrying about having to sort of reconnect a route. There’s some aspects of our private routing that could be made easier on developers. This is a big one. stuff like that.

It’s more visible. We’re starting to actively design the block store finally, which is going to be the thing that lets us store larger chunks of immutable data. um So you think about like movies and videos, where would you put those things?

Big, big files. You know, DHT is great for smaller things that change a lot. Mutable data that is small, so like the state of chat logs or state that states that change. um But it’s less good for like storing big blobs of stuff and moving it around, so torrenting, moving like big stuff between things and streaming.

Those things are going to be coming in our next few major releases.

That’s exciting.

And of course, a Veilid chat. I want to get that stabilized. To stabilize that, we need our DHT performing a little better we need this thing called rehydration.

Recalling rehydration is the mechanism by which if a replicated data set, you push a DHT record out, it goes to five different nodes. If two of those disappear, it needs to notice and push and replicate copies of the data so that it goes back up to five replicas. Right now, if nodes disappear in the middle, and the number of replicas drops, it is a more manual process than it should be to have it automatically replicate to other nodes. So again, that’s another feature that’s coming too that we have through design. It is coming as a second like the second thing after that that route auto-publishing.

That should stabilize Veilid chat pretty well, at which point, you know, we could do things like real time chat and not just sending messages back and forth, but also, um, being able to do things like, like web RTC style real time calling would become an option, VOIP, that kind of thing.

Last question, how can people stay up to date with Veilid?

Look at our release logs. Stay on the Discord. you know Mute it if you want. Every once in a while, pop in and look at the announcements. We are pretty I’m sure that at least a few times a month, we’re posting something of you know the new developments.

If you want a more technical perspective on the progress, you can go to our GitLab issues. We have issues and we track all of our work through there. And you can see what issues are being opened and closed and and the discussions that we have.

Link:
https://gitlab.com/veilid/veilid/-/issues

We have an RFC process now that is a formal request for comment style, like how to add big new features to Veilid. So we have gone through that process one time. It’s brand new. We will be doing it more. But if you want to see how we’re evolving the architecture of Veilid following the RFC repository is a very low volume way of participating and seeing how we’re thinking about architecture and having architectural conversations.

Link:
https://gitlab.com/veilid/rfcs

If you go to GitLab dot.com slash Veilid, you’ll see all of our projects. And there’s one called RFCs in there. ah There’s the Veilid repo. Obviously there’s the Veilid chat repo.

Those are the big ones that are of interest.

I can say that, we(Cult of the Dead Cow) do try to show up at a lot of conferences myself and medus4, and the Gibson and, TC or super T’s on, on discord. We all go and talk about veil it a lot.

If you want to come intercept us at a conference, please do. we love to talk about it. We love to get people involved. Some of our best contributors we’ve hung out with them at conferences and had a some great discussions about what value could look like in the future and it’s influenced our direction too. This is a community. It’s not just an open source project. It’s a community. By all means, reach out to us. We’re friendly. We want people to help us.

Spread the word and contribute any way you can. let’s set like let’s make the future happen here.

if I’m a conference organizer and I want you guys to come to my conference and talk about Veilid, like what is the best way to reach out?

Email is great because we have a record of it. If you go to, Veilid.com, there’s a contact section that will go right to an email. Otherwise press at veilid dot com.

We’re also on Twitter and also on Blue Sky. Yay. We’re also on Fettiverse. Yay.

So feel free to reach out to us on our various social media accounts.

Audio link: #

https://soundcloud.com/podcast-117246687/rustcareers-interview-with-veilids-lead-rust-developer-dildog

External links: #

https://veilid.com
https://veilid.org/
https://veilid.com/discord
https://gitlab.com/veilid/
https://gitlab.com/veilid/rfcs
https://en.wikipedia.org/wiki/WebRTC
https://en.wikipedia.org/wiki/Christien_Rioux
https://en.wikipedia.org/wiki/Cult_of_the_Dead_Cow
https://bevyengine.org/
https://docs.rs/async-std/latest/async_std/
https://async.rs/
https://tokio.rs/
https://en.wikipedia.org/wiki/The_Tor_Project
https://en.wikipedia.org/wiki/InterPlanetary_File_System
https://docs.rs/try/latest/try/macro.try.html
Deprication of the try! macro
https://kimundi.github.io/owning-ref-rs/owning_ref/index.html
https://users.rust-lang.org/t/whats-the-current-state-of-the-owning-ref-crate/102098

https://en.wikipedia.org/wiki/Dart_(programming_language)
https://blog.rust-lang.org/2014/09/15/Rust-1.0.html