Skip to main content

S3 SDK with Rust (Manual Credential Configuration)

· 5 min read
Huakun Shen
Website Owner

First of all, I have to say, I am very disappointed with AWS's documentation. They do have many documentation and sample code, but I am still unable to find what I was looking for (easily).

Intro

I was working on a project that requires using Rust to upload files to AWS S3. I wanted to use Rest API to do this, but could not find enough information from the documentation. There is no sample code or something like a postman API doc that allows you to generate client code from a Rest API.

For example, in this API doc on PutObject, https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html.

Authorization:authorization string doesn't mean anything to me. I have access key and secret, but there must be a way to get this authorization string. I am pretty sure it exists, and must be somewhere in the docs. I just couldn't find it. Put a link in the documentation isn't hard. It saves people time from looking through your entire documentation. The purpose of World Wide Web is to link things together, instead of look for things separately and try to assemble in clients' head.

Then I switched to Rust SDK. They have plenty of documentation and sample code; but I got stuck on one problem for a long time. Again, authorization. The documentation and sample code always assume you have the same scenario as they do. They assume you have a ~/.aws/credentials file with your access key id and secret. Sample code always loads credentials automatically from default locations or environment variables, which is fine for a server application. For client-side software, this doesn't hold. I need to explicitly pass credentials to a function to generate a client. This is possible and documented for both Python and Nodejs version of the doc, but not for Rust.

I had to go over so many documentation and sample code to figure out how to do this naive thing. Function from another Rust crate (package) has to be used. aws_types.

Basically, there are many different ways to produce credentials and client; but for someone without prior knowledge about your nasty design, there is no way to know which package I should find what the method needed. If you decide to put things in different packages, then at least provide an obvious link somewhere to indicate "You have the option to do blah blah, read the docs here".

Reading AWS docs (Rust) is like browse information everywhere and try to assemble in my head. Without enough prior knowledge, it's not easy to get things done quickly.

Details

Comparison

When I google "AWS s3 python client credential loading", the first link gives me what I need: Passing credentials as parameters . Took me 10 seconds to find the answer.

For Nodejs, it took me ~10 minutes. To find docs and examples everywhere. This is how I found the solution eventually.

  1. Google "aws s3 create nodejs client with credentials"
  2. Found S3 Client - AWS SDK for JavaScript v3, the JS package API docs
  3. S3Client class API
  4. Constructor API
  5. S3ClientConfig Interface API
  6. Properties -> Credentials Type
  7. AwsCredentialIdentity (Type of credentials property)
  8. Finally found that this is where to pass in accessKeyId, expiration, secretAccessKey, sessionToken.

This is no different from browsing source code. It's important developers has the ability to read source code and API docs. That doesn't mean the docs provider don't need to provide easy access to the most basic functionalities.

At least I could figure out Nodejs solution within 20 minutes. Took me a few hours to figure out the Rust solution.

Final Words

  • Also, why is documentation and examples everywhere? aws.amazon.com, github.com, and external websites like S3 Client - AWS SDK for JavaScript v3, (different for every language).
  • It's OK to have external API docs as each language have their own platforms. Like rust docs for rust crates.
  • But you should have a central place for links to everywhere and a easy-to-use search utility.
  • Could you put everything in one place and provide a search utility to search everything?
  • Like what you have in JS API docs
  • If your example is on GitHub, it's not that to search through the source code

Windows OpenSSH Server authorized_keys Failure on Administrative User Account

· 2 min read
Huakun Shen
Website Owner

Problem

I run into a problem related to ssh and got stuck for a few days. On a fresh-intsalled win11 PC, I tried to configure OpenSSh server, which I have done a thousand times on Linux OS.

I could ssh into the windows PC, but only using password; Key-based auth (i.e. password-less) doesn't work no matter what I do.

I added the authorized_keys file to C:\Users\username\.ssh, checked the file permission and even created another user account to compare.

None of these fixed the problem, and I couldn't find a good solution from Google or Chat GPT. BTW, Chat GPT keeps giving me Linux-related solutions.

Solution

The solution is actually quite simple. I read the official Doc by microsoft. Key-based authentication (Administrative user).

It turns out, Windows treat admin account differently. On Linux, regular accounts and sudoer accounts both use $HOME/.ssh/authorized_keys to stored trusted public keys.

The reason of my failure was because I was trying to ssh into an admin account.

On Windows, a administrators_authorized_keys should be placed under C:\ProgramData\ssh\. The content of administrators_authorized_keys is exactly the same.

Solution verified to work.

Comment

  • I can't blame Microsoft as this may be a more secure approach, and it's clearly documented in their documentation.
    • But maybe don't hide this at the bottom of the page. Highlight it in the beginning. It's easy to ignore it.
  • However, I don't understand why I can't easily find solution for such a common topic online. One would not search with the "administrative" keyword for such problem. Maybe Windows should give more warning messages and hints when OpenSSH server is installed or when login failed.
  • Furthermore, Chat GPT still has a long way to go.
  • And Google Search, ... hope you don't get completely replaced by some tech like Chat GPT one day.

Good Luck Hacking!

Fix Windows Network Adapter Low Speed

· 3 min read
Huakun Shen
Website Owner

This is a very weird bug. I will describe the scenario first, then the hacky fix.

Scenario

Environment

Description

I built a new PC with Gigabyte Z690 UD AX DDR4. The wifi 6 wireless adapter works fine, reaches 800+Mbps on my gigabit network; but the ethernet connection can only reach ~10Mbps.

This is super weird, the name is Realtek Gaming 2.5GbE Family Controlleras, the ethernet adapter should support 2.5G, and there is nothing wrong with the router.

Debug Process

  • I plugged in a 2.5G usb C ethernet converter, it worked fine.
  • When I use Ubuntu, the ethernet works fine (reach 900+Mbps).

From this debugging process, we know that the hardware should be fine. The problem most likely comes from Windows, or compatibility issue between windows and the network adapter (driver).

Solution

V1

The first time I got it working is by opening "Device", uninstall the device, and reinstall it.

After scanning, the network adapter is back, and the network speed is fixed (900+Mbps).

But this process has to be done every time a computer is booted.

V2 (auto)

This powershell script basically automate the previous solution using powershell.

caution

It requires Adminitrator permission to run the script.

Running this manually everytime is not better than the previous solution, we have to auto-run it on system starts.

The commented out lines is for debugging purpose, to see if the script has really run.

caution

Edit the following script to fit your scenario.

e.g. Update the deviceName, and log path.

# echo "start" | Out-File -Append C:\Users\user\Desktop\debug.log
# date | Out-File -Append C:\Users\user\Desktop\debug.log
# Start-Sleep -Seconds 5
$deviceName="Realtek Gaming 2.5GbE Family Controller"
foreach ($dev in (Get-PnpDevice | Where-Object{$_.Name -eq $deviceName})) {
&"pnputil" /remove-device $dev.InstanceId
}
pnputil /scan-devices
# echo "finish" | Out-File -Append C:\Users\user\Desktop\debug.log
  1. Press Win key, search and open "Task Scheduler"
  2. Action -> Create Basic Task Wizard
  3. Give a name
  4. Use "When I log on" for Trigger
  5. Action should be "Start a program", then fill in the script path
    1. Enter powershell <script-path>
  6. Then Finish
  7. Go to Task Scheduler Library -> Your Task, check Run with highest priviledges

Restart the computer, it should now be fixed.

If it doesn't, you may need to debug.

  1. Check the variables in script
  2. Run the script in powershell as Administrator and see if the network speed is normal.
  3. Then restart and see
  4. If it still doesn't work, uncomment the commented out code to see if the code has really run
  5. If it still doesn't work, then I don't know, good luck
Blame

Microsoft, Realtek, and Gigabyte, I don't know which of you should be responsible for this bug, but this needs to be fixed.

A gaming motherboard can't game at 10Mbps!

I personally believe Microsft/Windows should take the blame, as it works fine on Ubuntu and the driver itself should be fine. Reinstalling the device fixes the problem could induce that a wrong driver is used (override the correct driver).

This solution is hacky and inelegant, but I can't find a better solution online.

Come on, Microsoft.