feat(hosting): Implement Digital Ocean hosting support #140

Merged
NaiJi merged 60 commits from digital-ocean into master 2022-11-23 15:28:33 +02:00
Collaborator

Added:

  • picker between Hetzner and Digital Ocean on initializing page.
  • picker for server location
  • picker for server type
  • digital ocean api support alongside hetzner
Added: - picker between Hetzner and Digital Ocean on initializing page. - picker for server location - picker for server type - digital ocean api support alongside hetzner
NaiJi added the
Feature request
label 2022-10-23 04:34:54 +03:00
inex was assigned by NaiJi 2022-10-23 04:34:54 +03:00
NaiJi self-assigned this 2022-10-23 04:34:54 +03:00
NaiJi added 21 commits 2022-10-23 04:34:54 +03:00
f40ed08b02 feat(volume): Implement volume endpoints for Digital Ocean
volumeId type in VolumeApiProvider interfaces is now replaced with String from int to support Digital Ocean's UUID notation
Poster
Collaborator

cc @inex
What is needed to be done now to get rid of WIP status:

  • Review UI pages and related changes
  • Design Digital Ocean support for recovery pages
cc @inex What is needed to be done now to get rid of WIP status: - Review UI pages and related changes - Design Digital Ocean support for recovery pages
NaiJi requested review from inex 2022-10-23 04:37:31 +03:00
NaiJi changed title from WIP: feat: Implement Digital Ocean hosting support to WIP: feat(hosting): Implement Digital Ocean hosting support 2022-10-23 04:38:30 +03:00
NaiJi added 1 commit 2022-10-25 11:22:55 +03:00
NaiJi added 1 commit 2022-10-26 19:08:05 +03:00
NaiJi added 1 commit 2022-10-26 19:42:03 +03:00
NaiJi added 1 commit 2022-10-26 20:27:43 +03:00
NaiJi added 2 commits 2022-10-27 16:09:30 +03:00
NaiJi added 1 commit 2022-10-27 19:04:50 +03:00
NaiJi added 1 commit 2022-10-29 11:04:38 +03:00
NaiJi added 2 commits 2022-11-02 17:00:36 +02:00
NaiJi added 2 commits 2022-11-04 14:02:30 +02:00
NaiJi added 1 commit 2022-11-04 14:06:01 +02:00
aa1c04fdb8 fix(digital-ocean): Fix response code checking on volume actions
We can't compare with 'compelted' because Digital Ocean responses with 'in progress' right away or something, so it's better to check if it's just not 'error'
NaiJi added 1 commit 2022-11-04 14:19:55 +02:00
59d186a8af fix(volume): Change raw int to DiskSize object in volume endpoints
We already have an abstraction for size, there is no need to pass size value as raw numeric variables
inex added 1 commit 2022-11-08 03:25:09 +02:00
NaiJi force-pushed digital-ocean from 7627b916fa to 657e4555d3 2022-11-09 18:47:30 +02:00 Compare
NaiJi force-pushed digital-ocean from 657e4555d3 to 8f3b2dd20a 2022-11-09 20:39:08 +02:00 Compare
NaiJi force-pushed digital-ocean from 8f3b2dd20a to bea2427590 2022-11-09 20:40:43 +02:00 Compare
NaiJi force-pushed digital-ocean from bea2427590 to cdc47ecdb3 2022-11-09 20:43:15 +02:00 Compare
NaiJi added 1 commit 2022-11-10 19:03:40 +02:00
b3395915da fix(digital-ocean): Add correct linuxDevice path to volume objects
linuxDevice consists of supposedly hardcoded 'scsi-0DO_Volume_' plus given volume name
NaiJi added 1 commit 2022-11-11 05:32:29 +02:00
10bdd4c800 refactor(server-api): Make general server info polymorphic
Removing Hetzner type hardcode from server page and replacing it with generic String-based metadata container
NaiJi added 1 commit 2022-11-11 13:31:31 +02:00
NaiJi force-pushed digital-ocean from c8298c7600 to a7cbde663e 2022-11-12 20:11:36 +02:00 Compare
NaiJi added 1 commit 2022-11-12 20:44:57 +02:00
e20063a9ad chore: Remove unneeded metrics fields
PPS metrics aren't used in our charts
NaiJi added 1 commit 2022-11-14 07:49:22 +02:00
NaiJi force-pushed digital-ocean from 4a07f36d14 to 6b90f83d0e 2022-11-15 05:47:05 +02:00 Compare
NaiJi force-pushed digital-ocean from 6b90f83d0e to a7d74f1efc 2022-11-15 05:48:50 +02:00 Compare
NaiJi force-pushed digital-ocean from a7d74f1efc to 661b5ca26f 2022-11-15 05:56:03 +02:00 Compare
NaiJi force-pushed digital-ocean from 661b5ca26f to 92b417a103 2022-11-15 06:00:24 +02:00 Compare
NaiJi added 1 commit 2022-11-15 22:50:01 +02:00
2a5fceae91 fix(hetzner): Fix endpoints urls
Incorrect dereferencing in strings
NaiJi added 1 commit 2022-11-16 02:14:19 +02:00
NaiJi added 1 commit 2022-11-16 02:25:18 +02:00
0234278c2c refactor: Rename server.dart to server_api.dart
Because the class is called ServerApi, not just Server, it's totally not consistent with all other apis
NaiJi added 1 commit 2022-11-17 09:15:06 +02:00
NaiJi added 1 commit 2022-11-17 09:22:16 +02:00
NaiJi added 1 commit 2022-11-18 03:20:28 +02:00
NaiJi changed title from WIP: feat(hosting): Implement Digital Ocean hosting support to feat(hosting): Implement Digital Ocean hosting support 2022-11-18 05:11:31 +02:00
Poster
Collaborator

@inex All requested business cases for Digital Ocean are implemented. Now we need to test everything and I heavily encourage you to join me on this quest, because the end result is CRUCIAL, I am afraid all alone I can't guarantee it'll be as good as expected, I may miss a lot of stuff due to amount of our functionality.

@inex All requested business cases for Digital Ocean are implemented. Now we need to test everything and I heavily encourage you to join me on this quest, because the end result is CRUCIAL, I am afraid all alone I can't guarantee it'll be as good as expected, I may miss a lot of stuff due to amount of our functionality.
NaiJi force-pushed digital-ocean from 3b2c04466e to 0c4da8eb9f 2022-11-18 07:49:29 +02:00 Compare
NaiJi added 1 commit 2022-11-18 09:00:15 +02:00
da394e22ac feat(server-api): Implement bad certificates fallback for staging environment
Without it client application won't accept staging certificates from server
NaiJi added 2 commits 2022-11-18 09:33:48 +02:00
b26e22cd4e fix: Check on null for server type identifier
It is not needed to finish installation so it's okat if it's empty
NaiJi added 1 commit 2022-11-20 12:48:48 +02:00
NaiJi added 1 commit 2022-11-20 16:31:58 +02:00
58c9e00ce0 fix(recovery): Add reverse dns validation for digital ocean
In Digital Ocean reverse dns is not domain name but just name, like mydomainname instead of mydomainname.xyz, so we need additional condition
NaiJi added 1 commit 2022-11-20 16:34:52 +02:00
NaiJi added 1 commit 2022-11-20 16:35:56 +02:00
NaiJi added 1 commit 2022-11-22 14:10:36 +02:00
inex requested changes 2022-11-22 17:12:59 +02:00
@ -270,3 +272,3 @@
"place_where_data": "Здесь будут жить ваши данные и SelfPrivacy-сервисы:",
"how": "Как получить API Token",
"hetzner_bad_key_error": "Hetzner API ключ неверен",
"provider_bad_key_error": "Provider API key is invalid",

Forgot the translation here, but in fact, we may just fill the en.json file and fill all other translations via weblate

Forgot the translation here, but in fact, we may just fill the en.json file and fill all other translations via weblate
NaiJi marked this conversation as resolved
@ -6,2 +19,4 @@
final httpLink = HttpLink(
'https://api.$rootAddress/graphql',
httpClient: IOClient(httpClient),

StagingOptions.stagingAcme ? IOClient(httpClient) : null

The default client looks slightly different, maybe we shouldn't override it?

`StagingOptions.stagingAcme ? IOClient(httpClient) : null` The default client looks slightly different, maybe we shouldn't override it?
NaiJi marked this conversation as resolved
@ -0,0 +95,4 @@
try {
final GraphQLClient client = await getClient();
response = await client.query$SystemServerProvider();

Use final and declare the type. Implication for some reason ignores our GraphQL model and we cannot use properly parsedData without type declaration.

final QueryResult<Query$SystemServerProvider> response = await client.query$SystemServerProvider();
Use final and declare the type. Implication for some reason ignores our GraphQL model and we cannot use properly parsedData without type declaration. ```dart final QueryResult<Query$SystemServerProvider> response = await client.query$SystemServerProvider(); ```
NaiJi marked this conversation as resolved
@ -0,0 +96,4 @@
try {
final GraphQLClient client = await getClient();
response = await client.query$SystemServerProvider();
if (response.hasException) {

let's do if (response.hasException || response.parsedData == null) { just in case

let's do `if (response.hasException || response.parsedData == null) {` just in case
Poster
Collaborator

Condition response.hasException checks for a graphql exception content to print(response.exception.toString()) it.

If hasException returns false, then response.exception.toString() won't do anything, it has nothing to do with parsedData.

Condition `response.hasException` checks for a graphql exception content to `print(response.exception.toString())` it. If `hasException` returns false, then `response.exception.toString()` won't do anything, it has nothing to do with `parsedData`.
@ -0,0 +99,4 @@
if (response.hasException) {
print(response.exception.toString());
}
final rawProviderValue = response.data!['system']['provider']['provider'];

Try not to access raw values. Using the constructor over parsedData will give us more type safety.

final providerValue = response.parsedData!.system.provider.provider;
providerType = ServerProvider.fromGraphQL(providerValue);

Example contructor code will be provided at the server_details.dart section.

Try not to access raw values. Using the constructor over parsedData will give us more type safety. ```dart final providerValue = response.parsedData!.system.provider.provider; providerType = ServerProvider.fromGraphQL(providerValue); ``` Example contructor code will be provided at the server_details.dart section.
NaiJi marked this conversation as resolved
@ -0,0 +56,4 @@
String get infectProviderName => 'digitalocean';
@override
String get appearanceProviderName => 'Digital Ocean';

More like displayName

More like `displayName`
NaiJi marked this conversation as resolved
@ -0,0 +95,4 @@
@override
Future<Price?> getPricePerGb() async => Price(
value: 0.10,
currency: 'USD',

As we have a Price class, why don't make a Currency class, to ensure type safety and get cool getters like

  • getShortcode
  • getFormat
  • getIcon
    etc
As we have a Price class, why don't make a Currency class, to ensure type safety and get cool getters like - getShortcode - getFormat - getIcon etc
Poster
Collaborator

It's beyond the scope of Digital Ocean implementation, though it indeed is better to be done asap, it will be planned.

It's beyond the scope of Digital Ocean implementation, though it indeed is better to be done asap, it will be planned.
@ -0,0 +102,4 @@
Future<ServerVolume?> createVolume() async {
ServerVolume? volume;
final Response dbCreateResponse;

Quite counterintuitive to have a linal variable like this and have it reassigned later, but anyways.

maybe it should be a createVolumeResponse?

Quite counterintuitive to have a linal variable like this and have it reassigned later, but anyways. maybe it should be a `createVolumeResponse`?
NaiJi marked this conversation as resolved
@ -0,0 +118,4 @@
'filesystem_type': 'ext4',
},
);
final dbId = dbCreateResponse.data['volume']['id'];

db → volume

db → volume
NaiJi marked this conversation as resolved
@ -0,0 +126,4 @@
name: dbName,
sizeByte: dbSize,
serverId: null,
linuxDevice: 'scsi-0DO_Volume_$dbName',

linuxDevice is a full path!

/dev/disk/by-id/scsi-0DO_Volume_$dbName

`linuxDevice` is a full path! `/dev/disk/by-id/scsi-0DO_Volume_$dbName`
NaiJi marked this conversation as resolved
@ -0,0 +154,4 @@
final List<dynamic> rawVolumes = dbGetResponse.data['volumes'];
int id = 0;
for (final rawVolume in rawVolumes) {
final dbId = rawVolume['id'];

Same as before, do not forget to rename every db in var names and fix linuxDevice

Same as before, do not forget to rename every `db` in var names and fix linuxDevice
NaiJi marked this conversation as resolved
@ -0,0 +178,4 @@
}
Future<ServerVolume?> getVolume(final String volumeUuid) async {
ServerVolume? neededVolume;

requestedVolume?

`requestedVolume`?
NaiJi marked this conversation as resolved
@ -0,0 +195,4 @@
Future<void> deleteVolume(final ServerVolume volume) async {
final Dio client = await getClient();
try {
await client.delete('/volumes/$volume.uuid');

You are not acessing uuid here. '/volumes/${volume.uuid}' must be used.

You are not acessing uuid here. '/volumes/${volume.uuid}' **must** be used.
NaiJi marked this conversation as resolved
@ -0,0 +450,4 @@
return server.copyWith(startTime: DateTime.now());
}
/// Digital Ocean returns a map of lists of /proc/state values,

It is /proc/stat, not state.

It is `/proc/stat`, not `state`.
NaiJi marked this conversation as resolved
@ -0,0 +500,4 @@
const int step = 15;
final Dio client = await getClient();
//try {

commented try block?

commented try block?
NaiJi marked this conversation as resolved
@ -0,0 +572,4 @@
metadata = [
ServerMetadataEntity(
type: MetadataType.id,
name: 'server.server_id'.tr(),

.tr() should be called by UI layer.

.tr() should be called by UI layer.
Poster
Collaborator

Will be a part of coming refactoring for business logic layer. Consider it a stub

Will be a part of coming refactoring for business logic layer. Consider it a stub
@ -0,0 +653,4 @@
return servers;
}
String? getEmojiFlag(final String query) {

I guess it shouldn't be here. Wouldn't the one function for all providers be better?

While I understand that DO has its own declarations for every flag, we might make it a .fromDigitalOceanSlug constructor on some location enum.

I guess it shouldn't be here. Wouldn't the one function for all providers be better? While I understand that DO has its own declarations for every flag, we might make it a .fromDigitalOceanSlug constructor on some location enum.
Poster
Collaborator

Will be a part of coming refactoring for business logic layer. Consider it a stub

Will be a part of coming refactoring for business logic layer. Consider it a stub
@ -373,3 +408,3 @@
if (!success) {
await Future.delayed(const Duration(seconds: 10));
await deleteVolume(dbId);
await deleteVolume(dataBase);

dataBase...

dataBase...
NaiJi marked this conversation as resolved
@ -26,0 +54,4 @@
/// Actual provider name to render on information page for user,
/// for example 'Digital Ocean' for Digital Ocean
String get appearanceProviderName;

displayName

displayName
NaiJi marked this conversation as resolved
@ -80,2 +83,4 @@
hetzner,
@HiveField(2)
digitalOcean,
}

Example of a factory:

@HiveType(typeId: 101)
enum ServerProvider {
  @HiveField(0)
  unknown,
  @HiveField(1)
  hetzner,
  @HiveField(2)
  digitalOcean;

  factory ServerProvider.fromGraphQL(final Enum$ServerProvider provider) {
    switch (provider) {
      case Enum$ServerProvider.HETZNER:
        return hetzner;
      case Enum$ServerProvider.DIGITALOCEAN:
        return digitalOcean;
      default:
        return unknown;
    }
  }
}
Example of a factory: ```dart @HiveType(typeId: 101) enum ServerProvider { @HiveField(0) unknown, @HiveField(1) hetzner, @HiveField(2) digitalOcean; factory ServerProvider.fromGraphQL(final Enum$ServerProvider provider) { switch (provider) { case Enum$ServerProvider.HETZNER: return hetzner; case Enum$ServerProvider.DIGITALOCEAN: return digitalOcean; default: return unknown; } } } ```
NaiJi marked this conversation as resolved
@ -0,0 +5,4 @@
});
double value;
String currency;

Currency should be a smart enum.

Currency should be a smart enum.
Poster
Collaborator

It's beyond the scope of Digital Ocean implementation, though it indeed is better to be done asap, it will be planned.

It's beyond the scope of Digital Ocean implementation, though it indeed is better to be done asap, it will be planned.
@ -2,2 +2,4 @@
class _TextDetails extends StatelessWidget {
final Map<MetadataType, IconData> metadataToIcon = const {
MetadataType.id: Icons.numbers_outlined,

Move these icons to the MetadataType enum, so we can access them as metadata.type.icon

The enum would look like this:

enum MetadataType {
  id(icon: Icons.numbers_outlined),
  status(icon: Icons.mode_standby_outlined),
  cpu(icon: Icons.memory_outlined),
  ram(icon: Icons.memory_outlined),
  // But! If there are dollars, we have to use a dollar sign.
  cost(icon: Icons.euro_outlined),
  location(icon: Icons.location_on_outlined),

  other(icon: Icons.info_outlined);

  const MetadataType({
    required this.icon,
  });

  final IconData icon;
}
Move these icons to the MetadataType enum, so we can access them as `metadata.type.icon` The enum would look like this: ```dart enum MetadataType { id(icon: Icons.numbers_outlined), status(icon: Icons.mode_standby_outlined), cpu(icon: Icons.memory_outlined), ram(icon: Icons.memory_outlined), // But! If there are dollars, we have to use a dollar sign. cost(icon: Icons.euro_outlined), location(icon: Icons.location_on_outlined), other(icon: Icons.info_outlined); const MetadataType({ required this.icon, }); final IconData icon; } ```
NaiJi marked this conversation as resolved
NaiJi added 1 commit 2022-11-23 09:56:05 +02:00
NaiJi requested review from inex 2022-11-23 09:57:37 +02:00
inex approved these changes 2022-11-23 14:17:01 +02:00
NaiJi merged commit 58ce0f0f8b into master 2022-11-23 15:28:33 +02:00
NaiJi deleted branch digital-ocean 2022-11-23 15:28:33 +02:00
Sign in to join this conversation.
No reviewers
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: SelfPrivacy/selfprivacy.org.app#140
There is no content yet.