Scaffold CLI - A Rust program to scaffold files

This blog post is about how I wrote a Rust program that can scaffold files based on a plan. It was made to help me speed up development at one of my clients, while learning Rust.

Table of contents

• Intro to the problem
• Planning a solution for the problem
• Setting up a Rust project
• Installing helpful packages
• Creating data structures
• Reading a plan from a file
• Executing the plan
• Outro

Intro to the problem

I was working with some frontend code in a client’s codebase. I ran into the issue of having to create 4 files every time I wanted to make a new component.

It was the same 4 files each time and I therefore thought: “Why not automate this?”

If I could create a small CLI-program that could generate/scaffold these files for each new component I needed to create, I would save a lot of time.

I have also been interested in learning some Rust recently as well and thought that this could be a good starter project to make something small and useful.

Planning a solution for the problem

When starting the design of the CLI tool, I didn’t want it to be specific to that problem, but a more generic scaffolding tool.

So what I came up with was to be able to create “plans” that the CLI program could execute. These plans would contain all details about what to generate and how to do it.

I decided to use TOML as the format for the plans and here is an example of what I came up with:

[meta]
name = "plan-name-here"
version = "0.1.0"

[inputs]
inputs = ["workDir", "componentName"]

[[files]]
template_file = "component.template.pug"
output_file = "{{workDir}}/frontend/my-project/components/{{componentName}}/{{componentName}}-component.pug"

[[files]]
template_file = "component-preview.template.pug"
output_file = "{{workDir}}/frontend/my-project/components/{{componentName}}/{{componentName}}-component-preview.pug"

So in the meta section, the name and version of the plan is presented.

The inputs section contains the dynamic properties needed to generate the files of the plan, for instance a component name. The idea is that the CLI will ask for these inputs before executing the plan.

Finally the files sections each represent a file in the plan. It contains an input template and an output file path, where to put the finished file. Output file paths can include values given as input from earlier, as seen in this example.

The plan files should be placed in a folder called plans and use the naming convention [plan name].plan.toml, where [plan name] is switched for the name of the plan. A folder with the same plan name should be created next to the plan file which should contain the template files used in the plan.

The idea is to be able to run the tool with just the name of a plan like this: scaffold-cli plan-name which would go look for a plan in the plans folder, read it and execute it.

Setting up a Rust project

With a basic plan for how the program should work out of the way, setting up a Rust project was next on the list.

The easiest way is to use Rustup, which can easily be downloaded and installed from the https://www.rust-lang.org/ website.

After this, to create a new project we open a terminal, navigate to the folder that should contain our project folder and we use the command cargo new scaffold-cli.

This creates a new folder called scaffold-cli with the basic files for building a Rust program.

We should now be able to navigate to that folder with the terminal and run cargo run to download dependencies and run the program to see that everything works.

Installing helpful packages

Instead of building out all features ourselves we can use some helpful Rust packages to speed up our development.

The packages we will use are:

• serde and serde_derive for serialising and deserialising Rust data structures.
• clap for handling CLI arguments in a typesafe way.
• toml for parsing TOML files.

They are easy to install with the cargo command, for instance cargo add clap.

Creating data structures

Now the time has come for us to write some code. We’ll start by defining our data structures used when reading in our plan.

First we want to create a type for our CLI arguments:

#[derive(Parser, Debug)]
struct Args {
    /// The name of the plan to execute
    plan_name: String,
}

Parser comes from the clap package and allows us to use this for type safe CLI arguments.

Debug is used as a simple means to make a type printable (formatted for debugging). This can for instance be helpful while testing the program to print values to the screen.

We only need the name of the plan, as the plan file will contain the rest.

The data structure for the plan looks like this:

#[derive(Deserialize, Debug)]
struct PlanConfig {
    meta: PlanMeta,
    inputs: PlanInputs,
    files: Vec<PlanFile>,
}

#[derive(Deserialize, Debug)]
struct PlanMeta {
    name: String,
    version: String,
}

#[derive(Deserialize, Debug)]
struct PlanInputs {
    inputs: Vec<String>,
}

#[derive(Deserialize, Debug)]
struct PlanFile {
    template_file: String,
    output_file: String,
}

The PlanConfig type is the overarching type that defines the plan’s TOML config file as a whole. It contains the sections meta, inputs and files.

It derives from Deserialize which comes from the serde package. This provides implementations to serialise or deserialise this type.

The other types defining the sections of the config file also derive from Deserialize as they all must be serialisable and deserialisable.

Note: We often see the use of the Vec type for lists of values in Rust.

Finally, we have a type used for storing Key Value pairs used to hold the input keys and the user provided input values together. This is used to exchange the keys in file content and paths with the user defined values.

#[derive(Debug)]
struct KeyVal {
    key: String,
    val: String,
}

Reading a plan from a file

Now that we’ve defined the types for the plan, we can define a function to read the plan.

As mentioned earlier, the input is only the name of the plan. We therefore need to create the full file path from the name of the plan.

We do this with the following code:

let mut plan_file_path = env::current_exe().unwrap();
plan_file_path.pop();
plan_file_path.push("plans");
plan_file_path.push(format!("{}.plan.toml", plan_name));

We start out finding the path to the executable. We then remove the executable file from the path, add the plans folder and then add the plan file, which should have the .plan.toml extension.

We can then convert the path to a string:

let plan_file_path_str = plan_file_path.to_str().unwrap();

In both these cases, I’ve used .unwrap() to get to the value. This could be rewritten with error handling instead (see below).

When we have the file path, we can start reading the content of the plan as a string. I’ve added some error handling here to provide better error messages.

let contents = match fs::read_to_string(plan_file_path_str) {
    Ok(contents) => contents,
    Err(_) => {
        eprintln!("Could not read file `{}`", plan_file_path_str);
        // Exit the program with exit code `1`.
        exit(1);
    }
};

What happens here is we call the fs::read_to_string function. It returns a Result which is an enum with an Ok type and an Err type. The Ok type also contains the result of the function call. The Err type contains an error object.

When we work with an enum, we can do matching on it, which we can do with the match keyword and a brackets containing the types of the enum.

In this example, when we match the Ok type, we just return the contents. If we match the Err type, we print an error message and then exits the program with an error code (not 0).

Many functions return the Result type, when the operation can throw errors, for instance I/O operations.

When the content is read, we can then use the toml package to parse the string into our data types.

let plan: PlanConfig = match toml::from_str(&contents) {
    Ok(plan) => plan,
    Err(err) => {
        eprintln!("Could not parse TOML file `{}`.", plan_file_path_str);
        eprintln!("Error: {:?}", err);
        // Exit the program with exit code `1`.
        exit(1);
    }
};

We give a reference to the content to the toml::from_str function to parse the content. Because we defined the variable to have type PlanConfig the compiler infers the generic parameter of the function.

The function also returns a result object and we match on that to return the plan upon success. We print a message when an error occurred and we also print the error object itself for more details.

Rust often makes use of references as the compiler has an ownership and borrowing mechanism.

We now have all the parts necessary to read in a plan. The whole function looks like this:

fn read_plan(plan_name: String) -> PlanConfig {
    let mut plan_file_path = env::current_exe().unwrap();
    plan_file_path.pop();
    plan_file_path.push("plans");
    plan_file_path.push(format!("{}.plan.toml", plan_name));

    let plan_file_path_str = plan_file_path.to_str().unwrap();

    let contents = match fs::read_to_string(plan_file_path_str) {
        Ok(contents) => contents,
        Err(_) => {
            // Write `msg` to `stderr`.
            eprintln!("Could not read file `{}`", plan_file_path_str);
            // Exit the program with exit code `1`.
            exit(1);
        }
    };

    let plan: PlanConfig = match toml::from_str(&contents) {
        Ok(plan) => plan,
        Err(err) => {
            // Write `msg` to `stderr`.
            eprintln!("Could not parse TOML file `{}`.", plan_file_path_str);
            eprintln!("Error: {:?}", err);
            // Exit the program with exit code `1`.
            exit(1);
        }
    };

    return plan;
}

Executing the plan

The execution of the plan occurs in two steps: getting the required inputs and creating the files.

The inputs are read from user input in the terminal and is used for injecting values into both paths and file content in the templates.

Reading the inputs

The plan lists the inputs required to execute the plan. To make it easy for the user to know what to type in when, we let the program write out to the terminal which input we are waiting for.

The program then waits for user input and creates a key-value pair of the input name and the user-given value. It is added to a list of all the inputs.

In Rust this looks like the following:

println!("Please provide the following inputs:\n");

let mut input_list = Vec::<KeyVal>::new();
plan.inputs.inputs.iter().for_each(|input_key| {
    println!("Input value for key: {:?}", input_key);
    let mut input = String::new();
    match io::stdin().read_line(&mut input) {
        Ok(_goes_into_input_above) => {}
        Err(_no_updates_is_fine) => {}
    }
    let input_val = input.trim().to_string();
    input_list.push(KeyVal {
        key: input_key.to_string(),
        val: input_val,
    });
});

We start by asking the user to write the values for the inputs. The inputs are then presented one at a time. We wait for user input from the terminal. This can throw an error, but we allow errors here as empty values are okay.

When the value has been loaded, we trim it to remove excess white space. A KeyVal instance is created with the input key and the value given by the user and it is added to a list.

Transpiling templates and paths

The second half of executing a plan is to create files from templates where certain content is replaced by the values the user indicated during the input phase.

We start by creating a function that transpiles a single file in our plan. It needs the plan name, the template file (of our type PlanFile) and the list of input key-value pairs.

As with reading the plan, we first create the path to the template file:

let mut template_dir = env::current_exe().unwrap();
template_dir.pop();
template_dir.push("plans");
template_dir.push(plan_name);

let template_file_path = template_dir.join(template.template_file.to_string());

We create the path to the executable, append the plans folder, then the folder for the specific plan, which is required to have the same name as the plan.

And finally we add the file name of the template.

We can then read in the content of the file like we also did for the plan:

let template_file_content = match fs::read_to_string(&template_file_path) {
    Ok(contents) => contents,
    Err(_) => {
        // Write `msg` to `stderr`.
        eprintln!("Could not read file: `{}`", template_file_path.display());
        // Exit the program with exit code `1`.
        exit(1);
    }
};

If an error occurs a message is printed that includes the template file path, so it is easier figure out, why an error occurred.

The transpiling itself is as simple as taking all the content of a file, loop over the input key-value pairs, find all placeholders and exchange them for the value the user gave us.

All placeholders are wrapped in curly braces {{placeholder}} (without spaces around). The placeholder word matches the input key.

We loop over all the key-value pairs in the inputs list and for each of them replace all key placeholders found with the corresponding value:

// Replace all keys with their values in the file's content.
let mut transpiled_file_content = template_file_content.to_string();
for kv in input_list {
    let key = format!("{{{{{}}}}}", kv.key);
    let val = kv.val.to_string();
    transpiled_file_content = transpiled_file_content.replace(&key, &val);
}

Since we want to change the template content we loaded, we need to have it as a mutable variable. This is marked by using the keyword mut (let mut transpiled_file_content).

If we don’t do this, the variable will be read-only, so we essentially copy the string into a mutable variable, so we can exchange the placeholders for the input values.

To create the placeholder string (using the curly braces) we create a new string using Rust’s format macro. It inserts the value (second parameter) into the string (first paramter) where it has a set of curly braces. Futhermore to write out a curly brace we need to escape it, so that is why we have so many curly braces in the format macro ({{ prints one curly brace, so we need two of these sets. The innermost open and close curly brace is what the format macro uses to insert the value and therefore we end up at 5 sets of braces).

As an example, format!("{{{{{}}}}}", "myKey") becomed "{{myKey}}".

Rust has a concept of macros which is a form of metaprogramming that expands the existing code.

After transpiling the content, we need to output the content into a file.

The output paths for the files we want to create can also contain placeholders, which means that we need to do the same for the output path before we can use it to write a new file to disk.

The code for creating a finished output path is the same as for the template content:

// Create output file path by replacing all keys with their values.
let mut output_file_path = template.output_file.to_string();
for kv in input_list {
    let key = format!("{{{{{}}}}}", kv.key);
    let val = kv.val.to_string();
    output_file_path = output_file_path.replace(&key, &val);
}

As an example of this, a path looking like this {{workDir}}/src/components/{{componentName}}.tsx could be turned into ./frontend/src/components/MyComponent.tsx.

For our next steps, we need the output path to be of the PathBuf type. This is easily made by the PathBuf::from function on the output_file_path string.

let output_path_buf = PathBuf::from(output_file_path);

Since the output path might include folders that aren’t created, we need to make sure all folders are created before we can create and write to the file.

Luckily there is a way to ensure all directories are created using the fs::create_dir_all function on a path of folders.

To get a path that only consists of folders (not containing the file name at the end) we can use the parent function on the output path PathBuf variable.

The parent function returns an Option type, which can either be a value or nothing. It works like the Result type in that it is an enum, but has the types Some with a value and None without a value.

The Option type is used in Rust instead of using null like in Java or C#.

Since we don’t need to do anything, when None is returned, we can shorthand the match on the enum by using the if let Some... construct:

if let Some(parent_dir) = output_path_buf.parent() {
    ... /* Do something */
}

This is equivalent to:

let parent_dir_exists = output_path_buf.parent() {
    Some(parent_dir) => ... /* Do something */,
    None => {}
}
if ()

So in the end the code looks like this:

// Create the output folders, if they don't exist.
if let Some(parent_dir) = output_path_buf.parent() {
    if let Err(error) = fs::create_dir_all(parent_dir) {
        eprintln!(
            "Failed to create folders for path '{}' : {}",
            parent_dir.display(),
            error
        );
        // Exit the program with exit code `1`.
        exit(1);
    }
}

We use the same construct inside, when calling the fs::create_dir_all, but with if let Err... to only do something when an error occurs.

When this piece of code has run, we are sure that all folders needed to create the file itself are present.

So to create a file, we use the fs::File::create function that takes an output path as input, creates a file and returns it.

We need this to be mutable, so we can add content to the file. We also add some error handling:

// Now create and write to the file.
let mut file = match fs::File::create(&output_path_buf) {
    Ok(file) => file,
    Err(err) => {
        eprintln!(
            "Failed to create file at '{}': {}",
            output_path_buf.display(),
            err
        );
        exit(1);
    }
};

The printing function can support several inputs, so here we have two sets of {}.

Having the file, we can now write content to the file using the write_all function. It can take a buf type as input, which we can get by calling as_bytes() on our content string:

if let Err(e) = file.write_all(transpiled_file_content.as_bytes()) {
    eprintln!(
        "Failed to write to file at '{}': {}",
        output_path_buf.display(),
        e
    );
    exit(1);
}

We only handle the error case, in which we prints out a message. Upon a success we continue.

To give some feedback to the user, we can finish by printing a message to the screen that the file has been created:

println!("Created file: {:?}", output_path_buf);

Using {:?} in print and format macros is intended to format for debugging. Some types can’t be output without a specific format indicator.

This will result in the whole function looking as follows:

fn transpile_and_save_template(plan_name: &String, template: &PlanFile, input_list: &Vec<KeyVal>) {
    let mut template_dir = env::current_exe().unwrap();
    template_dir.pop();
    template_dir.push("plans");
    template_dir.push(plan_name);

    let template_file_path = template_dir.join(template.template_file.to_string());

    let template_file_content = match fs::read_to_string(&template_file_path) {
        Ok(contents) => contents,
        Err(_) => {
            // Write `msg` to `stderr`.
            eprintln!("Could not read file: `{}`", template_file_path.display());
            // Exit the program with exit code `1`.
            exit(1);
        }
    };

    // Replace all keys with their values in the file's content.
    let mut transpiled_file_content = template_file_content.to_string();
    for kv in input_list {
        let key = format!("{{{{{}}}}}", kv.key);
        let val = kv.val.to_string();
        transpiled_file_content = transpiled_file_content.replace(&key, &val);
    }

    // Create output file path by replacing all keys with their values.
    let mut output_file_path = template.output_file.to_string();
    for kv in input_list {
        let key = format!("{{{{{}}}}}", kv.key);
        let val = kv.val.to_string();
        output_file_path = output_file_path.replace(&key, &val);
    }

    let output_path_buf = PathBuf::from(output_file_path);

    // Create the output folders, if they don't exist.
    if let Some(parent_dir) = output_path_buf.parent() {
        if let Err(error) = fs::create_dir_all(parent_dir) {
            eprintln!(
                "Failed to create folders for path '{}' : {}",
                parent_dir.display(),
                error
            );
            // Exit the program with exit code `1`.
            exit(1);
        }
    }

    // Now create and write to the file.
    let mut file = match fs::File::create(&output_path_buf) {
        Ok(file) => file,
        Err(err) => {
            eprintln!(
                "Failed to create file at '{}': {}",
                output_path_buf.display(),
                err
            );
            exit(1);
        }
    };

    if let Err(e) = file.write_all(transpiled_file_content.as_bytes()) {
        eprintln!(
            "Failed to write to file at '{}': {}",
            output_path_buf.display(),
            e
        );
        exit(1);
    }

    println!("Created file: {:?}", output_path_buf);
}

The last thing is to call this function for every file entry in the plan, which looks as follows:

plan.files.iter().for_each(|file| {
    transpile_and_save_template(&plan.meta.name, file, &input_list);
});

We create an iterator for the vector of file entries and for each of them, we call the function that transpiles and creates the file.

Putting it all together

We make use of all the above from our main function, which runs when the program is executed:

fn main() {
    let args = Args::parse();

    let plan: PlanConfig = read_plan(args.plan_name);

    println!(
        "Executing plan: {:?} - Version: {:?}\n---\n",
        plan.meta.name, plan.meta.version
    );

    println!("Please provide the following inputs:\n");

    let mut input_list = Vec::<KeyVal>::new();
    plan.inputs.inputs.iter().for_each(|input_key| {
        println!("Input value for key: {:?}", input_key);
        let mut input = String::new();
        match io::stdin().read_line(&mut input) {
            Ok(_goes_into_input_above) => {}
            Err(_no_updates_is_fine) => {}
        }
        let input_val = input.trim().to_string();
        input_list.push(KeyVal {
            key: input_key.to_string(),
            val: input_val,
        });
    });

    println!("\nCreating files...\n");

    plan.files.iter().for_each(|file| {
        transpile_and_save_template(&plan.meta.name, file, &input_list);
    });

    println!("\nFinished executing plan {:?}!", plan.meta.name);
}

We start by calling parse on our Args type. It has this function, because our type derived from the Parser trait. This will parse the arguments given to the executable and try to fit them in to the type we have defined.

We then read the plan given as argument using our read_plan function.

To give some user feedback, we print some information to the screen.

We then ask for the user input as described above after which we loop over all the files in the plan and call the transpile_and_save_template for each of them.

At the end we print another message to the screen informing the user that the plan has finished executing.

Outro

This is my first finished Rust program, so some of the code above can definitely be improved.

It did accomplish my goals, though, which were to learn more about Rust by writing a Rust program. It also helped me with my coding tasks, as creating these many files at once removed a very tedious and slow task.

The code is available in full on my GitHub and can be found here: https://github.com/phillipphoenix/scaffold-cli

In the repo, there is also an example plan. See more in the README file.