Activity

A feed of development activity from across Sia Foundation GitHub repositories.

Feb 20, 2024
  • The contract timeline labels have been darkened to increase contrast and readability.
Feb 20, 2024
  • The bandwidth settings for egress and ingress now have clearer descriptions.
Feb 20, 2024

That way a client doesn't need to generate a key themselves.

This PR also gets rid of the disablePreshardingEncryption flag and instead looks at the encryption key to determine whether encryption is enabled and an offset is necessary. This should offer better protection against misusing the multipart upload API. For everyone already using it correctly, this change is backwards compatible.

Feb 20, 2024

The "size" on the objects table is missing an index which slows down certain operations tremendously

Feb 19, 2024

Recently had to add a metric migration and found that we don't properly build the path in performMigration, it always uses main:

// build path
var path string
if isSQLite(db) {
    path = fmt.Sprintf("migrations/sqlite/main/migration_" + name + ".sql")
} else {
    path = fmt.Sprintf("migrations/mysql/main/migration_" + name + ".sql")
}

Tested it with a dummy metrics/main migration on my MySQL node.

Feb 19, 2024
Feb 19, 2024

Bumps github.com/minio/minio-go/v7 from 7.0.66 to 7.0.67.

Release notes

Sourced from github.com/minio/minio-go/v7's releases.

Bugfix release

What's Changed

New Contributors

Full Changelog: https://github.com/minio/minio-go/compare/v7.0.66...v7.0.67

Commits

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Feb 19, 2024

Bumps gorm.io/driver/mysql from 1.5.2 to 1.5.4.

Commits

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Feb 17, 2024

Simplifies the sector pruning code improving performance and readability

Feb 16, 2024

Reverts a change made in v1.0.2 that checks if the number of sectors deleted from the database matches the number requested. I haven't been able to discover the edge case that causes the logic to fail yet. The SELECT and DELETE happen in the same transaction, so they should all be present and the contract_sector_roots table doesn't have any external constraints that would be preventing deletion.

Closes #309

Feb 16, 2024
Feb 16, 2024
  • Integrity check alert sector data will now update when there is an ongoing integrity check.
  • All data will now refresh more frequently.
Feb 16, 2024

When writing the docs I had a slight change of heart. This PR merges the dismiss routes and makes "all" a query param.

Feb 16, 2024
Feb 16, 2024
Feb 16, 2024

This adds a new bCurrency type which will eventually replace currency and the 2 field currency we use right now. It's nicer because it requires only a single column and can still be used for comparisons or sorting if we ever need that.

The reason it was added now is because we are migrating the wallet db pretty soon anyway so we can make use of this type instead of keeping the old one.

Feb 16, 2024

This PR upgrades our coreutils dependency to v0.0.2. I'm only upgrading the dependency in this PR, I did not bother properly renaming events -> transactions since I've done that already in the single address wallet PR.

Feb 16, 2024

Moves the wallet logic out of the sqlite package and into the wallet package.

Feb 16, 2024
Feb 15, 2024
Feb 14, 2024

This PR cherry-picks a fix or a panic onto master

Feb 14, 2024

To avoid a potential deadlock where we never receive a response since the request gets queued but not processed

Feb 14, 2024

I discovered something quite nasty on my node, where I had a couple of critical downloads that were failing to migrate because we were applying the "MigrationSurchargeMultiplier" over and over again to the MaxDownloadPrice, eventually causing an overflow.

9e5bd369: failed to apply surcharge multiplier of 10 to 60.3 TS, overflow detected
DEBUG PJ: setting adjust max dl price 6.03 KS
DEBUG PJ: setting adjust max dl price 60.3 KS
...
DEBUG PJ: setting adjust max dl price 603 GS
DEBUG PJ: setting adjust max dl price 6.03 TS
DEBUG PJ: setting adjust max dl price 60.3 TS

Since that is the worst thing to happen to a critical migration I figured it'd be best to a) extend the gouging settings validation to ensure a user never enters a combination that overflows and b) stop preventing a potential overflow (which is now no longer possible) from actually executing the critical download

I'm on the fence whether we need to extend the bus checks on startup, we might be invalidating gouging settings in the wild, although I feel that's extremely unlikely... we could handle the error and revert only the multiplier to the default setting. If it were to happen, we currently revert the user's gouging settings to the default values.

Feb 14, 2024
2024-02-13T21:48:42.216288715Z panic: no interaction recorder attached to the context
2024-02-13T21:48:42.216340814Z 
2024-02-13T21:48:42.216344851Z goroutine 2280 [running]:
2024-02-13T21:48:42.216348238Z go.sia.tech/renterd/worker.HostInteractionRecorderFromContext(...)
2024-02-13T21:48:42.216351103Z     /renterd/worker/interactions.go:48
2024-02-13T21:48:42.216354429Z go.sia.tech/renterd/worker.(*host).FetchPriceTable.func1.1({0x1a408e8, 0xc003dbec40}, 0xc20ff046e0?)
2024-02-13T21:48:42.216357465Z     /renterd/worker/host.go:199 +0x305
2024-02-13T21:48:42.216360621Z go.sia.tech/renterd/worker.(*transportPoolV3).withTransportV3(0xc000a7f2f0, {0x1a408e8, 0xc003dbec40}, {0x3c, 0x51, 0x24, 0x5a, 0x69, 0xb4, 0x87, ...}, ...)
2024-02-13T21:48:42.216363567Z     /renterd/worker/rhpv3.go:225 +0x175
2024-02-13T21:48:42.216366472Z go.sia.tech/renterd/worker.(*host).FetchPriceTable.func1(_)
2024-02-13T21:48:42.216369237Z     /renterd/worker/host.go:197 +0xca
2024-02-13T21:48:42.216372303Z go.sia.tech/renterd/worker.(*host).FetchPriceTable(_, {_, _}, _)
2024-02-13T21:48:42.216376090Z     /renterd/worker/host.go:216 +0x1ed
2024-02-13T21:48:42.216379467Z go.sia.tech/renterd/worker.(*priceTable).fetch(_, {_, _}, _)
2024-02-13T21:48:42.216383334Z     /renterd/worker/pricetables.go:160 +0x717
2024-02-13T21:48:42.216386630Z go.sia.tech/renterd/worker.(*priceTables).fetch(_, {_, _}, {0x3c, 0x51, 0x24, 0x5a, 0x69, 0xb4, 0x87, ...}, ...)
2024-02-13T21:48:42.216390517Z     /renterd/worker/pricetables.go:86 +0x213
2024-02-13T21:48:42.216394605Z go.sia.tech/renterd/worker.(*host).priceTable(_, {_, _}, _)
2024-02-13T21:48:42.216398242Z     /renterd/worker/rhpv3.go:517 +0xf7
2024-02-13T21:48:42.216402250Z go.sia.tech/renterd/worker.(*host).fetchRevisionWithAccount.func1.1.1(_)
2024-02-13T21:48:42.216407860Z     /renterd/worker/rhpv3.go:286 +0x65
2024-02-13T21:48:42.216413040Z go.sia.tech/renterd/worker.RPCLatestRevision({_, _}, _, {0x56, 0xc5, 0xd5, 0x3d, 0x49, 0xb6, 0xc5, ...}, ...)
2024-02-13T21:48:42.216441123Z     /renterd/worker/rhpv3.go:716 +0x228
2024-02-13T21:48:42.216444249Z go.sia.tech/renterd/worker.(*host).fetchRevisionWithAccount.func1.1({0x1a408e8?, 0xc003dbec40?}, 0xc0043e9860?)
2024-02-13T21:48:42.216447725Z     /renterd/worker/rhpv3.go:285 +0xdb
2024-02-13T21:48:42.216452204Z go.sia.tech/renterd/worker.(*transportPoolV3).withTransportV3(0xc000a7f2f0, {0x1a408e8, 0xc003dbec40}, {0x3c, 0x51, 0x24, 0x5a, 0x69, 0xb4, 0x87, ...}, ...)
2024-02-13T21:48:42.216455450Z     /renterd/worker/rhpv3.go:225 +0x175
2024-02-13T21:48:42.216458145Z go.sia.tech/renterd/worker.(*host).fetchRevisionWithAccount.func1()
2024-02-13T21:48:42.216461151Z     /renterd/worker/rhpv3.go:284 +0xb3
2024-02-13T21:48:42.216463826Z go.sia.tech/renterd/worker.(*account).WithWithdrawal(0xc005643400, {0x1a408e8?, 0xc003dbec40?}, 0xc016617400)
2024-02-13T21:48:42.216467943Z     /renterd/worker/rhpv3.go:446 +0x2b6
2024-02-13T21:48:42.216470749Z go.sia.tech/renterd/worker.(*host).fetchRevisionWithAccount(_, {_, _}, {0x3c, 0x51, 0x24, 0x5a, 0x69, 0xb4, 0x87, ...}, ...)
2024-02-13T21:48:42.216474786Z     /renterd/worker/rhpv3.go:282 +0xf9
2024-02-13T21:48:42.216477511Z go.sia.tech/renterd/worker.(*host).FetchRevision(0xc005657080, {_, _}, _)
2024-02-13T21:48:42.216480237Z     /renterd/worker/rhpv3.go:254 +0x1a6
2024-02-13T21:48:42.216483423Z go.sia.tech/renterd/worker.(*uploader).execute(0xc00565afc0, 0xc05ae45f10)
2024-02-13T21:48:42.216491137Z     /renterd/worker/uploader.go:205 +0x305
2024-02-13T21:48:42.216500174Z go.sia.tech/renterd/worker.(*uploader).Start(0xc00565afc0)
2024-02-13T21:48:42.216503440Z     /renterd/worker/uploader.go:112 +0x165
2024-02-13T21:48:42.216507057Z created by go.sia.tech/renterd/worker.(*uploadManager).refreshUploaders in goroutine 119
2024-02-13T21:48:42.216510864Z     /renterd/worker/upload.go:756 +0x6ea

If the only option for an unset interaction recorder is to panic, that seems like a bad fit for context.Value

Feb 13, 2024
  • Files can now be moved by dragging into or out of directories.
  • Files and directories can now be renamed via the context menu.
Feb 13, 2024

Tried to enable goleak in renterd in our integration tests and it kept failing due to the registry recorder not waiting for a shutdown. I thought it might make sense to add it to hostd too.

Feb 12, 2024
Feb 12, 2024

Additional test for the metadata store that focuses on slabs getting reused for multiple objects.

Feb 12, 2024

Bumps gorm.io/driver/sqlite from 1.5.4 to 1.5.5.

Commits

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Feb 12, 2024
Feb 12, 2024

Bumps golang.org/x/term from 0.16.0 to 0.17.0.

Commits

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Feb 12, 2024

Bumps golang.org/x/crypto from 0.18.0 to 0.19.0.

Commits

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Feb 12, 2024

I noticed that most of the churn on Arequipa comes from host being offline while other nodes report they are not. To rule out scans being fragile this PR scans twice if necessary.

Ran this on Arequipa and immediately after startup I'm seeing renterd-arequipa | 2024-02-09T13:49:43Z INFO worker.worker worker/worker.go:1430 successfully scanned host ed25519:ec7389996f3cf39fe5073b82d98c65145936863082d6af16a307a6180b750cfc after retry so it seems like it's making things a bit more robust.

Feb 12, 2024

Bumps go.sia.tech/core from 0.1.12-0.20231211182757-77190f04f90b to 0.2.1.

Commits

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Feb 10, 2024
Feb 10, 2024
Feb 9, 2024
  • Uploading nested directories now preserves structure rather than flattening files into current directory.
  • Files can now be sorted by size.
Feb 9, 2024
Feb 9, 2024
Feb 8, 2024
Feb 8, 2024
Feb 8, 2024
Feb 8, 2024
Feb 8, 2024

Updated prometheus encoder to include additional metrics for siagrafana hostd dashboard. Metrics previously outputting as stringified currencies are now outputting as Siacoins(). Corrected a few typos in other metrics.

Feb 8, 2024
Feb 7, 2024
Feb 7, 2024

This will trigger actions in SiaFoundation/linux as well as SiaFoundation/sia-homebrew every time hostd publishes a version that's not an rc version.

Feb 6, 2024

Adds additional verification to check that sector roots modified during a revision match the expected state

Feb 6, 2024
  • 90813d49: Removed the maximum registry size configuration option.
Feb 6, 2024
Feb 6, 2024
Feb 5, 2024

Add golangci-lint and fix a couple issues it found. I used the config file currently in hostd. I think the linters are a bit nitpicky (they say having banned and Banned on EphemeralPeerStore is confusing) but I didn't disagree enough to make an exception with //nolint.

Feb 2, 2024
Feb 1, 2024
Feb 1, 2024

This effort is part of grant project

Introduces a startup param --prometheus to enable various /prometheus endpoints

Please see https://github.com/SiaFoundation/hostd/pull/288 for info on refactor considerations

Best regards

Feb 1, 2024

Submitting an update to the prometheus endpoints for hostd.

  1. the current encoder for hostd is not compatible with the hostd dashboard in siagrafana repo
  2. the current encoder for hostd shouldn't include timestamps. prometheus is a tool for tracking and measuring various metrics, it does this by periodically checking the system's status, and the specific timing of these checks is usually not critical. it's important to note that prometheus is primarily designed for monitoring purposes, rather than serving as a general purpose time series database.
  3. this rolls back the url param and reintroduces a startup param --prometheus to enable /prometheus endpoints which is what the siagrafana scripts are currently generating prometheus configs with.
  1. the current hostd encoder should be leveraged in a future refactor of the hostd fork.
  2. regarding the url params. i did leave comments on this approach very early in the project and never received feedback https://github.com/SiaFoundation/hostd/pull/213 . so i can include in the refactor switching to something like url param if that is still desired.
  3. submitting the PR more as a formality for closing the grant than wanting to merge before refactor
Jan 31, 2024
Jan 31, 2024
  • Removed the maximum registry size configuration option.
Jan 30, 2024
Jan 30, 2024

The current encoding of Currency uses an 8-byte length prefix followed by a variable-length big-endian encoding of the integer. This is inherently wasteful, as currency values never require more than a 1-byte prefix. Variable-length encodings can also hurt performance in certain use cases, and make it harder to predict the real-world size of various encoded payloads.

This PR adds an alternative fixed-width (16-byte) little-endian encoding for Currency. This matches the actual struct definition of Currency in core: two uint64s. Surprisingly, this is not quite as efficient as simply reducing the 8-byte prefix to a 1-byte prefix; the currency values currently present in the blockchain would encode to between 11-15 bytes on average. Still, the advantages of fixed width outweigh the slightly larger size imo.

As a side effect, I also ended up changing the encoding of SiafundOutput to be more compact in v2.

Jan 30, 2024
Jan 27, 2024
  • Fixed an error where looking up a host was showing an error page for unscanned hosts.
Jan 26, 2024
  • 8f8a17f3: Update the net address placeholder value to show port 9982.
Jan 25, 2024

Update the net address placeholder value to show port 9982.

Jan 25, 2024
  • 4dbb448d: Fixed an error with "lastAnnouncement" that occurs the first time the configuration is saved.
Jan 24, 2024
  • The file stats now only show health once there is more than 0 bytes of file data.
  • Fixed an error with "lastAnnouncement" that occurs the first time the configuration is saved.
Jan 24, 2024
Jan 23, 2024
Jan 17, 2024
  • S3 authentication keypairs can now be created and managed directly from the UI.
Jan 16, 2024
Jan 16, 2024
  • Fixed an issue where the app would not navigate when the metrics data interval was set to ALL.
Jan 16, 2024

Adds an internal Prometheus encoder for encoding responses in the Prometheus sample format.

Also adds a response path param to some endpoints to support different output formats (i.e. http://localhost:9980/api/settings?response=prometheus)

Supported formats:

  • prometheus
  • json

Supported endpoints:

  • GET /state/host
  • GET /state/consensus
  • GET /settings
  • GET /metrics
  • GET /wallet
Jan 16, 2024

hostd settings is showing incorrect number for base rpc price and Sector access price. It's off by 10 due to using 1e7 instead of 1e6 for calculating millions

Jan 16, 2024
Jan 16, 2024

I thought newcomers could use more guidance on development practices for this repo. Added changeset, commit, and test instructions under the development section.

Jan 15, 2024
Jan 12, 2024
Jan 12, 2024
Jan 11, 2024
Jan 11, 2024
Jan 10, 2024
Jan 10, 2024

Adds a hostd config command to generate a hostd.yml config file based on user input.

Jan 6, 2024

check() already calls log.Fatal if err != nil, so this code is redundant

Jan 5, 2024
Jan 3, 2024

This effort is part of grant project

Added various GET endpoints to return metrics formatted for Prometheus.

Prometheus metrics endpoint is disabled by default and runs on its own port.

Jan 3, 2024
  • 0fc1c36c: Fixed an issue with the copy to clipboard feature.
Jan 3, 2024
Dec 15, 2023
  • Copyable entity values now have a context menu with support for opening Siascan pages.
Dec 12, 2023
  • The send siacoin and siafund features now support specifying custom change and claim addresses.
  • Ledger wallets now support sending siafunds.
  • Seed wallets now support sending siafunds.
  • Event balances are now calculated with only relevant transaction components.
  • The ledger generate addresses dialog now shows a close action if no new addresses have been generated.
  • Currency display can now be configured to siacoin, fiat, or both along with a preference for when only one can be displayed.
  • App preferences no longer list unused GPU setting.
Dec 11, 2023

We ran into bugs related to determining the right amount of collateral when renewing contracts. Most notably exceeding the max collateral setting of hosts.

There were multiple reasons for that. I was able to get rid of most errors by postponing computing the collateral until after we have acquired a contract.

The other issue, which is the reason for this PR, is the fact that we used ContractRenewalCollateral to compute the newCollateral and then CalculateHostPayouts to create the payouts and both of these functions perform the same math but slightly different. (one uses fc.EndHeight() and the other one fc.WindowEnd` for determining the extension of the renewal).

Since we don't use rhpv2 renewals I'm not going to bother refactoring that but I'm updating rhpv3 to make sure there is no duplicate code anymore and that everything, including the capping of the collateral according to the host's MaxCollateral happens in one place.

Right now this is deployed on Arequipa and there are no more collateral errors.

Dec 9, 2023

I have been adding this to our repos so we can do things like check struct tags automatically. I know core is still changing so if this is unwanted right now please feel free to close this PR.

Edit: Need to do some Ledger review fixes today. Will fix remaining warnings tomorrow.

Dec 8, 2023

I noticed renewal failures in our logs that indicate we're not properly respecting the host's MaxCollateral setting.

{"level":"error","date":"2023-12-07T16:56:27Z","component":"autopilot.contractor","msg":"renewal failed, err: couldn't renew contract: RPCRenew: failed to read RPCRenewContractHostAdditions: ReadResponse: failed to validate renewal: collateral exceeds maximum: expected at most 10000000000000000000000000000 got 10000047475326975980269993984\n","hk":"ed25519:c4b451fa7ab7085763119aeb1aa34db524c5dc6416dc77c5ba82100eb9687790","fcid":"fcid:5a32d8bbfb7e6a29f874ad2ded500d46e3e9aa4675db624b7f782b7fd7f45e05"}
Nov 30, 2023

Apparently, the workflow needs to be on the branch to trigger

Nov 29, 2023
Nov 29, 2023
Nov 3, 2023

I finished working on these tests last weekend, but due to unforeseen circumstances, I couldn't submit the pull request. Today I can finally do it, but I noticed that just yesterday another contributor, who coincidentally worked on some of the same test features as I did, submitted his PR #132

I wasn't sure how to proceed in this situation, so I decided to run test coverage. In my case, the overall coverage percentage was 77.3%, while @chris124567's was 79.8%, which I congratulate him on. However, if we combine our PRs, the rate reaches 80.1%

Chris and I approached the task and code-writing differently, and to be honest, I'm not sure how you'll handle our PRs after comparing and analyzing them; nonetheless, I decided to offer alternative tests for the Marshaler and Unmarshaler functions and submit this PR.

Nov 3, 2023

Refers #130

Nov 3, 2023

Get consensus coverage up to 79.8%

Oct 20, 2023
  • Wallet now shows siafund balances when greater than 0.
Oct 16, 2023

This is a slightly bigger change, so I figured I'd PR it to allow for some discussion instead of just pushing it.

Sia computes a lot of hashes. When I started implementing v2, I took care to make hashing as fast as possible. In particular, I wanted hashing operations to be zero-alloc. That meant accepting some ugliness like hasherPool and eschewing helper functions that might not be inlined.

This PR essentially reverses that decision. Instead of every ID function grabbing a hasher and directly calling EncodeTo on all the stuff we want to encode, we now pass those things to a magic hashAll helper function. The resulting code is definitely more readable; you can see at a glance exactly what's being hashed. The downside is that all of our hashes now allocate, making them a bit slower. How much slower? Anywhere from 1% to 15%, depending on how little data is being hashed. Is that acceptable? Well, when I run our test suite 100x, it takes basically the same amount of time -- definitely nowhere near 15% slower. Regardless -- preventable allocations make me sad. At least ~all of our hashing will be performed inside hashAll now, which means CPU profiles will show us how much time we spend hashing in general...?

Another consequence is that the exact data we're hashing has changed slightly. Basically, instead of omitting signatures, we now hash empty signatures. Why? Because it means instead of writing this:

func (s State) ContractSigHash(fc types.V2FileContract) types.Hash256 {
	return hashAll(
		"sig/filecontract",
		s.v2ReplayPrefix(),
		fc.Filesize,
		fc.FileMerkleRoot,
		fc.ProofHeight,
		fc.ExpirationHeight,
		fc.RenterOutput,
		fc.HostOutput,
		fc.MissedHostValue,
		fc.RenterPublicKey,
		fc.HostPublicKey,
		fc.RevisionNumber,
	)
}

we can write this:

func (s State) ContractSigHash(fc types.V2FileContract) types.Hash256 {
	fc.RenterSignature, fc.HostSignature = types.Signature{}, types.Signature{}
	return hashAll("sig/filecontract", s.v2ReplayPrefix(), fc)
}

Having a helper like hashAll doesn't force us to make this change, but it sure makes it a lot more compelling, doesn't it?

Lemme know if you have strong feelings about this either way. Frankly, I'm pretty confident that this is the right choice; writing a big post about it is probably just my way of coping with the loss of my precious zero-alloc code 😭

Oct 13, 2023

Add a test similar to TestP2P in walletd and add some extra cases mostly relating to file contracts to the validation test.

Oct 11, 2023

I was working around this in both renterd and hostd but it's easier to update it here instead.

Oct 4, 2023

This PR adds the rpcCost type from hostd. We can move it over to core and get rid of some of the very rough estimates for proof sizes and replace them by exact values we get from DiffProofSize.

Oct 2, 2023

Fixes the node12 deprecation warning

Oct 2, 2023

Increases the go versions used in CI from 1.17 and 1.18 to 1.20 and 1.21

Sep 22, 2023

Signs Windows releases to satisfy Windows smart screen.

Sep 7, 2023

This removes a few random whitespaces, and fixes the build script for docker so it works on forked repos. It'll still obviously be the same for this repo too.

Sep 6, 2023
  • Wallet seeds now get cached after an address is successfully generated.
  • Ledger and seed wallets should now use the correct fee when sending transactions with or without the include fee option.
  • walletd now supports Ledger-based wallets.
  • Ledger wallets now support generating a specific address or a range of addresses via Ledger device.
  • Outputs are now properly released when a transaction fails during signing or broadcast.
  • The GPU setting now displays the correct device support text.
  • Fixed an issue with the syncing status staying at 99%.
  • Addresses can now be deleted from the context menu.
  • Both ledger and seed wallets now save public keys in address metadata, this is especially important to streamline Ledger signing.
  • Support for connecting Ledger devices over USB, HID or Bluetooth depending on browser support.
Aug 29, 2023

Adds validation tests for V2Blocks/V2Transactions.

Aug 25, 2023
Aug 10, 2023

Not sure if this is the right place to put this considering that the return type is a boolean, but some kind of safeguard against this seems appropriate.

Aug 9, 2023
  • The app now warns the user if it is running on the testnet with a prominent banner.
  • Table headers now freeze in view when the table scrolls.
  • Fixed an issue where walletd would indefinitely report as "syncing" - the synced state is now based on the most recent block's timestamp being within the last 2 hours.
  • The contrast was improved on the syncing progress text in the daemon profile.
Aug 7, 2023

Prevent an unnecessary 1 KB allocation from being repeated in all the put calls.

Jul 22, 2023

Changes boltDB interface in node.go to match changes in https://github.com/SiaFoundation/core/pull/121

Jul 19, 2023

This began as an effort to make the DB commit less frequently, which is important for speeding up IBD. Currently, we commit (i.e. fsync) after every block, which means hundreds of thousands of fsyncs are required for a full IBD -- often many fsyncs per second. Needless to say, this is enormously wasteful. Ideally, we would fsync once every few seconds, and/or after buffering a large number of writes. So that's what this PR does: it tracks how long it's been since the last fsync, and how many writes we've buffered since then, and only flushes after we cross a threshold.

In the course of implementing this, I realized that our DB interface was more powerful than necessary. Specifically, it supports fully-isolated transactions, meaning you can have multiple "views" of the DB open concurrently, and those views won't observe each others effects, and can be committed or rolled back independently. However, our usage of the DB is already serialized by the chain.Manager's mutex, and clients of chain.Manager only have limited access to the DB. So we actually don't need transactions; all we need is a way to force an fsync, so that we can guarantee that a subscriber will never commit data that hasn't been committed by the manager itself. That's what this PR does.

This still needs to be tested and benchmarked on an actual node, so I'll hold off on merging until then. Hopefully we'll see a significant speedup!

Jul 18, 2023
  • Currency options now include AUD.
  • The wallet list now has a column that shows the lock status. If the wallet is unlocked, the tooltip also shows the remaining time.
  • The wallet list now has a siacoin balance column.
  • Wallet dropdown now includes unlock, edit, and delete operations.
  • Wallet auto-locking can be enabled or disabled from the App Preferences menu - the feature is enabled by default. The locking inactivity period can also be configured.
  • Seed-based wallets can now be unlocked for a period of time. Each wallet will lock after a period of inactivity or if the app is closed.
  • The Settings dialog is now called App preferences.
  • Operations that require a seed, such as sending siacoin or generated addresses now use cached seed values when a wallet is already unlocked.
  • The app now includes an auto-lock feature that can be enabled or disabled from the App Preferences menu. The locking inactivity period can also be configured.
  • The login screen now has an option to show the custom API field.
  • The login screen timeout for an unresponsive daemon is now 10 seconds.
Jul 1, 2023
Jun 30, 2023
Jun 27, 2023
May 30, 2023

hostKey is passed to PrepareContractRenewal, although it is not used there.

May 29, 2023
May 25, 2023

To avoid accidentally dialing localhost in case the port is set but the address isn't.

May 5, 2023
// MarshalText implements encoding.TextMarshaler.
func (uk UnlockKey) MarshalText() ([]byte, error) {
	return marshalHex(uk.Algorithm.String(), uk.Key[:])
}

Causes UnlockKey to JSON encode as "\"ed25519\":48a468d62e48e27ef24707bd1f467476e9605954e201834c659e614b9704d041"

This will likely break a bunch of things in hostd and renterd. Is adding compatibility code for this in core worth it?

Apr 28, 2023
Apr 28, 2023

This PR adds the InitProgramHelper which makes it a bit nicer to compute costs for programs.

e.g. computing the cost of a read sector program becomes this

rc := pt.InitProgramCost()
rc.Add(pt.ReadSectorCost(rhpv2.SectorSize))
cost, _ := rc.Total()

where previously we couldn't make use of the ResourceCost type for the base cost.

Also snuck in a receiver change that Nate proposed.

Apr 20, 2023

This should prevent an overflow panic when the hosts set their prices ridiculously high. Could fix renterd/#283

Apr 18, 2023
Apr 18, 2023

This came up in a renterd review the other day so I decided to propose it. I'm still not sure whether it's silly or actually useful so I figured I would let you be the judge Luke. Definitely feel free to just close this out, I was expecting way more occurrences to be honest, was surprised to only find a handful.

Apr 18, 2023

This PR adds a Name field to the Network object in the consensus package. By adding the network name the UI can differentiate between the mainnet and the testnet and adjust accordingly.

Apr 13, 2023

Turns out if you have a MarshalText method, you don't need MarshalJSON too. So that's nice.

I still want to write a JSON encoder at some point; it would be really nice if we could autogenerate fast encode/decode functions, with options like useCamelCase and noNullSlices -- but for now, struct tags will have to do.

Apr 1, 2023

This is a broad refactor that removes hot-wallet functionality from walletd. The new plan is for walletd to be a watch-only server, like walrus. Address generation and transaction signing will be performed by clients, namely the built-in UI.

This PR also includes a syncer implementation. This is a core piece of any Sia node, and will (eventually) be copied into renterd and hostd. It also includes a txpool, but really just the stub of one, not a proper implementation. This is the last piece of functionality needed to fully migrate to core, so it's my top priority right now.

So, why push incomplete code? Simple: it's so that @alexfreska can look at the API endpoints and start working on a UI. 😉

Feb 28, 2023

Delete wallet/bip39.go. Missed this yesterday but it's not necessary anymore since this logic is in core now.

Feb 27, 2023

Use the seed functions, StandardAddress, StandardUnlockConditions from core/wallet and remove the copies in wallet/types.go