GSoC 2022: Progress Report 2

Hello everyone. It is finally possible to run the whole Rust testing suit for UEFI under QEMU and OVMF. So I think this is a good point to give a detailed overview of everything that has been implemented and the state of those implementations. I will also provide instructions on how to go about running the tests as well.


Running Rust Tests

Setup

We will follow the instructions in Running Tests in a Remote Machine.

  1. Clone my fork of Rust source and switch to uefi-std-rebase branch:
git clone https://github.com/Ayush1325/rust.git
cd rust
git checkout uefi-std-rebase
  1. Make sure that all the dependencies have been met. Try building std for UEFI:
./x.py build --stage 1 --target x86_64-unknown-uefi
  1. Build remote-test-server:
./x.py build --stage 1 --target x86_64-unknown-uefi src/tools/remote-test-server

Launch QEMU

Now we have to launch the generated build/x86_64-unknown-linux-gnu/stage1-tools-bin/remote-test-server.efi in QEMU. The following options are needed for launching this executable:

  1. Network with port 12345 forwarded
-netdev user,id=net0,hostfwd=tcp::12345-:12345 -device virtio-net-pci,netdev=net0,mac=00:00:00:00:00:00
  1. OVMF:
-drive if=pflash,format=raw,file={Path to OVMF_CODE.fd},index=0 -drive if=pflash,format=raw,file={Path to OVMF_VARS.fd},index=1
  1. UEFI Shell: Not supplying this ISO makes OVMF try to boot from the network first for me, so I do this manually as well:
-drive format=raw,file={Path to UefiShell.iso},index=2
  1. Image containing the executable and a startup.nsh (for automatic startup): I would recommend not using vvfat since it gives a lot of mapping errors in intensive File I/O which is needed in running the tests

All of this can be simplified by using my fork of uefi-run. If you are using that, then it is possible to use the config I am using currently:

qemu-uefi build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-uefi/release/remote-test-server.efi -s 1024 --shell-path UefiShell.iso --vars-path OVMF_VARS.fd -b OVMF_CODE.fd --output-path output.txt -- -netdev user,id=net0,hostfwd=tcp::12345-:12345 -device virtio-net-pci,netdev=net0,mac=00:00:00:00:00:00

The output-path argument passes -serial output.txt since the stderr output is not visible on the VGA buffer for me. However, the output.txt is extremely useful for debugging since all the stderr (Rust panics and aborts) are present in the file with the correct location of panics.

I would also recommend launching the executable with verbose argument.

Run test

Any of the Rust test/s or test folders can be run on the QEMU instance using the following command:

TEST_DEVICE_ADDR="localhost:12345" ./x.py test src/test/ui/{FILE or Directory} --target x86_64-unknown-uefi --stage 1

Std Implementation Status

  • alloc
    • GlobalAlloc: Currently uses hardcoded EFI_MEMORY_TYPE::EfiLoaderData. Can be changed.
      • alloc
      • dealloc
  • args
    • Args
    • args: Implement using EFI_LOADED_IMAGE_PROTOCOL
  • cmath: Provided by compiler_builtins for UEFI.
  • env: Just contains some constants
  • fs
    • File
      • *open: Can only open files in the same volume as executable.
      • file_attr
      • *fsync: Uses EFI_FILE_PROTOCOL.Flush() internally
      • *datasync: Uses fsync internally
      • truncate
      • read
      • *read_vectored: Using rust default implementation.
      • is_read_vectored
      • write
      • *write_vectored: Using rust default implementation.
      • is_write_vectored
      • flush: Don’t really maintain any buffer in Rust side.
      • seek
      • duplicate
      • set_permissions
    • FileAttr
      • size
      • perm
      • file_type
      • modified
      • accessed
      • created
    • ReadDir
    • DirEntry
      • path
      • file_name
      • metadata
      • file_type
    • OpenOptions
      • new
      • read
      • write
      • append
      • truncate
      • create
      • create_new
    • FilePermissions
      • readonly
      • set_readonly
    • FileType
      • is_dir
      • is_file
      • *is_symlink: Just returns false since I don’t think UEFI supports symlinks.
    • DirBuilder
      • new
      • *mkdir: Can only open files in the same volume as executable.
    • readdir
    • unlink
    • rename
    • set_perm
    • rmdir
    • remove_dir_all
    • try_exists
    • readlink
    • symlink
    • link
    • stat
    • lstat
    • canonicalize
    • copy
  • *io: Using the default implementation at unsupported/io.rs. It works fine.
  • *locks: Using the default implementation at unsupported/locks.rs. It should work for any target having just a single-thread.
  • *net: Only implmented TCPv4 right now.
    • TcpStream
      • connect
      • connect_timeout
      • set_read_timeout
      • set_write_timeout
      • read_timeout
      • write_timeout
      • peek
      • read
      • *read_vectored: Using the default implementation right now. However, EFI_TCP4_PROTOCOL supports vectored read.
      • is_read_vectored
      • write
      • *write_vectored: Using the default implementation right now. However, EFI_TCP4_PROTOCOL supports vectored write.
      • is_write_vectored
      • peer_addr
      • socket_addr
      • *shutdown: Only implemented complete shutdown right now.
      • duplicate
      • linger
      • set_nodelay
      • nodelay
      • set_ttl
      • ttl
      • take_error
      • set_nonblocking
    • TcpListener
      • bind
      • socket_addr
      • accept
      • duplicate
      • set_ttl
      • ttl
      • set_only_v6
      • only_v6
      • take_error
      • set_nonblocking
    • UdpSocket
  • os
    • errno
    • error_string
    • getcwd: Returns the Text representation of EFI_DEVICE_PATH_PROTOCOL. This can be directly converted to EFI_DEVICE_PATH_PROTOCOL using the EFI_DEVICE_PATH_FROM_TEXT_PROTOCOL.
    • chdir
    • SplitPaths
    • split_paths
    • JoinPaths
    • join_paths
    • current_exe: Returns the Text representation of EFI_DEVICE_PATH_PROTOCOL. This can be directly converted to EFI_DEVICE_PATH_PROTOCOL using the EFI_DEVICE_PATH_FROM_TEXT_PROTOCOL.
    • Env
    • env
    • *getenv: Currently using a static Guid. Maybe should use the Guid used by UefiShell for Environment Variables.
    • setenv
    • unsetenv
    • temp_dir
    • home_dir
    • exit
    • getpid
  • os_str: Basically just UTF-8 strings. This is because os_str is supposed to be the superset of both te OS specific string(UCS-2) and Rust strings (UTF-8).
    • Buf
      • from_string
      • with_capacity
      • clear
      • capacity
      • reserve
      • try_reserve
      • reserve_exact
      • try_reserve_exact
      • shrink_to_fit
      • shrink_to
      • as_slice
      • as_mut_slice
      • into_string
      • push_slice
      • into_box
      • into_arc
      • into_rc
    • Slice
      • from_u8_slice
      • from_str
      • to_str
      • to_string_lossy
      • to_owned
      • clone_into
      • into_box
      • empty_box
      • into_arc
      • into_rc
      • make_ascii_lowercase
      • make_ascii_uppercase
      • to_ascii_lowercase
      • to_ascii_uppercase
      • is_ascii
      • eq_ignore_ascii_case
  • path
    • MAIN_SEP_STR
    • MAIN_SEP
    • is_sep_byte
    • is_verbatim_sep
    • parse_prefix
    • absolute
  • pipe
    • *AnonPipe: Implemented using Runtime Variables
      • read
      • read_vectored: Using default implementation
      • is_read_vectored
      • write
      • write_vectored: Using default implementation
      • is_write_vectored
      • diverge
    • *read2: It is blocking and synchronous right now
  • process
    • Command
      • new
      • arg
      • env_mut
      • cwd
      • *stdin: Only null working yet.
      • *stdout: Using my primitive AnonPipe
      • *stderr: Using my primitive AnonPipe
      • get_program
      • get_args
      • get_envs
      • get_current_dir
      • *spawn: Currently calling EFI_BOOT_SERVICES.StartImage() here since the pipes don’t really work asynchronously right now.
    • StdioPipes
    • Stdio
    • ExitStatus
      • exit_ok
      • code
    • ExitStatusError
      • code
    • ExitCode
      • as_i32
    • Process
      • id
      • kill
      • wait
      • try_wait
    • CommandArgs
  • stdio
    • *STDIN_BUF_SIZE: Currently using the same value as Windows
    • Stdin
      • new
      • *read: Buffered reading not implemented yet.
    • Stdout
      • new
      • write
      • flush
    • Stderr
      • new
      • write
      • flush
  • thread: Using unsupported/thread.rs
  • thread_local_key: Using unsupported/thread_local_key.rs
  • time
    • Instant
    • SystemTime: Implemented using GetTime()
      • now
      • sub_time
      • checked_add_duration
      • checked_sub_duration

Conclusion

Since the first main object to run Rust tests has now been met, I will work on improving/refactoring most of the implementations before trying to merge them to master. It would also be great if more people try out this std and give feedback.

Consider supporting me if you like my work.