14:30:58 <dkliban> #startmeeting Pulp Triage 2019-11-26
14:30:58 <dkliban> !start
14:30:58 <dkliban> #info dkliban has joined triage
14:30:58 <pulpbot> Meeting started Tue Nov 26 14:30:58 2019 UTC.  The chair is dkliban. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:30:58 <pulpbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:30:58 <pulpbot> The meeting name has been set to 'pulp_triage_2019-11-26'
14:30:58 <pulpbot> dkliban: dkliban has joined triage
14:31:12 <dkliban> This is the Content Signing in Pulp 3 meeeting.
14:31:35 <dkliban> despite what it says in the channel topic
14:31:38 <dkliban> https://etherpad.net/p/Pulp_-_Signing_Use_Cases
14:32:38 <dawalker> #info dawalker has joined triage
14:32:38 <dawalker> !here
14:32:38 <pulpbot> dawalker: dawalker has joined triage
14:32:45 <ttereshc> #info ttereshc has joined triage
14:32:45 <ttereshc> !here
14:32:45 <pulpbot> ttereshc: ttereshc has joined triage
14:33:08 <daviddavis> #info daviddavis has joined triage
14:33:08 <daviddavis> !here
14:33:08 <pulpbot> daviddavis: daviddavis has joined triage
14:33:13 <bmbouter> #info bmbouter has joined triage
14:33:13 <bmbouter> !here
14:33:14 <pulpbot> bmbouter: bmbouter has joined triage
14:33:16 <ppicka> #info ppicka has joined triage
14:33:16 <ppicka> !here
14:33:16 <pulpbot> ppicka: ppicka has joined triage
14:33:39 <dkliban> If  you have any additional use cases that we should discuss, please add them to the etherpad now
14:34:37 <dkliban> I would like to go through the use cases in the etherpad in the order that they are currently presented in
14:34:48 <bmbouter> great
14:34:54 <dkliban> So we will begin with the plugin writer experience
14:35:00 <bmbouter> also we can look for "high level" gpg actions, e.g. verify, sign, etc
14:36:00 <dkliban> bmbouter: line 3 has some assumptions built into it
14:36:09 <bmbouter> yes it does
14:36:21 <daviddavis> yea, I was going to ask about where this list of keys comes from
14:37:11 <bmbouter> x9c4: can chime in to correct me, but he told me his vision was that pulpcore would store public keys in "generic" facilities
14:37:29 <bmbouter> and he asked if he should write that, and I said he should write what he wanted to share and we can determine if we want to do that together
14:37:56 <bmbouter> so these key ids I believe are referenes to "saved GPG keys", almost likepulp is a public keyring manager
14:38:12 <bmbouter> he was considering it even as an independant plugin to do GPG keyring saving/management
14:38:15 <x9c4> I could imagine the keys to be organized in a gpg-plugin.
14:38:23 <x9c4> as usual content.
14:39:30 <bmbouter> so aside from how it's built, the general idea I'm reading is "key storage" and the crux is public keys only
14:39:47 <bmbouter> but there are other use cases further down where I think we need to store private keys
14:39:57 <bmbouter> we'll come to those later tho
14:40:18 <daviddavis> what about pointing pulp to a pre-existing gpg key storage so we don't have to manage keys?
14:41:59 <dkliban> daviddavis: is there one you have in mind?
14:42:01 <bmbouter> so let's look at the use cases because it's how I arrived at my conclusion
14:42:18 <daviddavis> no, I'm a bit of a novice when it comes to gpg and stuff
14:42:28 <bmbouter> it's ok we can learn together
14:42:31 <daviddavis> s/a bit of//
14:42:39 <bmbouter> let's consider our user personas
14:42:44 <daviddavis> ok
14:42:49 <x9c4> misa has tought me about why signing in pulp_deb (2) is done with a shell script. He has an external signing service and does _not_ save private keys.
14:43:14 <bmbouter> we have the administrator, this is the person you call when your pulp is broken
14:43:37 <bmbouter> and then we have many users, imagine a multi-tenant system, multiple teams etc
14:44:12 <bmbouter> RBAC eventually will help us sort this big pool of users into various sub-personas, e.g. the person who can sign repo X or the access the "gpg trust store" y
14:44:50 <bmbouter> a "gpg trust store" is the term I'm using for a "gpg home folder" which is the folder containing the various gpg data sources, this is where the keyring, public, and private keys would be stored (if you asked gpg to store such things)
14:45:24 <bmbouter> x9c4: +1 to not using these facilities and having a "call out" option for use cases like misa's
14:45:39 <bmbouter> but providing these facilities for the more common uses
14:46:07 <bmbouter> anyway, the question I keep asking myself is: "does the administrator have the key or does a user with the right RBAC permissions have the key"
14:46:10 <bmbouter> the private key
14:46:47 <dalley> apologies for the connection issues, I wanted to say something and then couldn't because my nick wasn't whitelisted and then had to reconnect and it took several minutes...
14:46:49 <bmbouter> if the administrator has the key, they would configure this in the settings folder, perhaps just pointing to one or more pre-configured "gpg trust stores" and pulp is then out of the key management space
14:46:56 <dalley> if this has already been addressed, my apologies
14:47:08 <dalley> if it's content, you're not implying that the "key" content would be present in the repository? I'm not sure we would want to do that
14:47:49 <misa> Sorry for being late
14:47:52 <dkliban> dalley: we have not addressed that. separate repos that contain public keys.
14:48:03 <misa> #info misa has joined triage
14:48:03 <misa> !here
14:48:03 <pulpbot> misa: misa has joined triage
14:48:08 <x9c4> #info x9c4 has joined triage
14:48:08 <x9c4> !here
14:48:08 <pulpbot> x9c4: x9c4 has joined triage
14:48:18 <misa> oops, not sure that's what I needed to do
14:48:23 <bmbouter> it is actually
14:49:01 <dkliban> so we are currently discussing how a plugin writer could validate signatures on content that it is syncing or processing during upload
14:49:02 <bmbouter> I think we only want pub and/or priv keys in so much as plugin writers want to use them for signing, verification, etc operations
14:49:56 <bmbouter> although a future gpg-plugin would be cool, for the first-cut I think we only want enoiugh to serve ^ plugin writer goal
14:50:26 <mikedep333> #info mikedep333 has joined triage
14:50:26 <mikedep333> !here
14:50:26 <pulpbot> mikedep333: mikedep333 has joined triage
14:50:54 <dkliban> that requires having access to some public keys. i am not too concerned about how pulp stores those public keys, but the plugin writer needs be able to access them in a programatic way.
14:51:32 <dkliban> this part is easy ... but the use cases that require using a private key to sign an artifact are the ones that give me pause
14:51:38 <bmbouter> yup I think there are two main needs and we'll need both, one is the "built in signing" that's the common one most environments will use
14:52:09 <bmbouter> the other is misa's use cases which is all the same verify/sign/etc operations only they are called to a system outside of pulp to perform
14:52:49 <dkliban> bmbouter: i don't think verifying needs to happen in an outside system ... misa?
14:53:17 <misa> Finished reading the thread. My biggest concern is how hard it is to design a generic system that can support whatever the user might want
14:53:30 <misa> dkliban: correct, verification can be built-in
14:53:53 <misa> dkliban: essentially, everything that touches secret material may be outsourced to an external service
14:54:01 <x9c4> But you don't want your content verified against a key, another user loaded into pulp.
14:54:41 <misa> x9c4: I think that's what I'm getting at. The RBAC-like syntax for this is confusing
14:54:58 <misa> "accept packages signed by this set of key ids" is the easy case
14:55:37 <x9c4> right.
14:55:38 <misa> what if they want "accept package foo signed by this key id, and package bar signed by this other key id"
14:55:41 <bmbouter> we need simple facilities for RBAC ownership of specific objects I think this fits nicely with that
14:56:09 <misa> (never mind that deb doesn't do per-package signatures - but I digress)
14:57:12 <bmbouter> I can envision how to build a thin layer around the gpg python bindings to have pulp handle this along w/ RBAC
14:57:30 <bmbouter> I cannot envision how to layer on "call out" to misa's signing service in a generalized way
14:57:50 <misa> bmbouter: let me try to eliminate my confusion: are we talking specifically about syncing in this use case?
14:58:06 <dkliban> yes, we are still on that
14:58:14 <x9c4> I guess, that could be a call to a suid binary
14:58:16 <iballou> I
14:58:47 <misa> external service is only needed for *signing*, not verified, right?
14:58:57 <x9c4> The arguments are "keyid to use" and "file to sign" and "mode..."
14:59:14 <misa> s/verified/verifying/
14:59:39 <ttereshc> misa, that's my understanding
14:59:54 <dkliban> yeah ... bmbouter, i would like to move on to lines 5 and 6 ... those deal with signing content
14:59:55 <bmbouter> mine too
15:00:11 <bmbouter> dkliban: +1
15:00:15 <misa> Depending on what you mean by artifact
15:00:31 <dkliban> Artifact is the most basic unit in Pulp 3
15:00:42 <misa> I don't think (re)signing packages should be Pulp's job
15:00:46 <dkliban> each file is an Artifact ... the same Artifact can belong to different content units
15:00:59 <bmbouter> misa: for pulp_ansible that is the use case we want
15:01:09 <bmbouter> for pulp to actually perform artifact signing as requested by plugin code
15:01:32 <misa> let me clarify. If pulp is not responsible with creating an artifact, it should not be responsible with signing the artifact
15:01:48 <misa> an rpm package is not created by pulp, it is uploaded to pulp
15:02:08 <misa> a yum repository is created by pulp from multiple rpm packages, so if metadata signing is needed, then pulp must do it
15:02:21 <ttereshc> I guess it would be a plugin writer's choice
15:02:23 <bmbouter> sure for pulp_ansible collection content also, it's uploaded to pulp, and then at some time later it is signed
15:02:36 <ttereshc> whether to support it or not
15:02:50 <dkliban> misa: the metadata for RPM repo is considered an artifact
15:03:00 <dkliban> and the plugin writer needs facilities to sign that artifact
15:03:32 <misa> dkliban: which is fine, the distinction I got to is at "who created it" level. Not sure if this is helping though.
15:03:55 <dkliban> it will be up to the plugin author to make that distinction
15:04:49 <dkliban> so when the plugin writer does need to sign an artifact, how will the plugin writer do that? and i heard 2 possible ways
15:04:56 <bmbouter> I think pulp_ansible actually wants to sign a file inside of the Artifact (just to make it more confusing)
15:05:09 <dkliban> bmbouter: oh wow
15:05:39 <bmbouter> yeah it's the MANIFEST of the tarball's contents not the tarball itself
15:05:47 <x9c4> And the signed file will be the new artifact?
15:06:10 <bmbouter> right now they want to offer via a viewset
15:06:26 <bmbouter> pulp_ansible already has a pretty chatty API between pulp and the CLI (the Galaxy V3 API)
15:07:35 <ttereshc> x9c4, I think it has to be, it will have different checksum and so on
15:07:50 <ttereshc> at least I hope we are not updating/changing the existing one
15:07:54 <dkliban> can we take a step back from the pulp_ansible use case? i would like to dicuss whether Pulp can avoid storing private keys
15:07:56 <x9c4> That MANIFEST could be the second artifact on the collection content unit...
15:08:51 <misa> May I offer this as for starting the discussion: https://github.com/pulp/pulp_deb/blob/2-master/README.md#signing-support
15:09:38 <misa> A similar kind of API, not just for Release (which is an artifact), but to any artifact
15:10:22 <bmbouter> misa: how much of this do you see coming from the settings versus from users?
15:10:36 <misa> most should be settings, I'd think
15:10:45 <misa> instance-level or repo-level settings
15:11:04 <misa> plugin writer could extend the signing infrastructure to expose their own variables
15:11:20 <misa> so that the signing infrastructure could make decisions on it
15:11:41 <misa> maybe things like PLUGIN_TYPE would also be exposed to the signing infrastructure
15:12:10 <misa> that'd be part of pulpcore, but the plugin author may specify extras as I was hinting
15:12:25 <bmbouter> one of the things I'm struggling with is that the users who need this are not the same people who setup pulp for pulp_ansible at least
15:12:46 <misa> bmbouter: there is a third persona, I think
15:12:55 <misa> repository admin
15:13:05 <misa> admin, repository admin, user
15:13:13 <bmbouter> yup that makes sense
15:13:15 <misa> users are read-only relative to content
15:13:23 <misa> admins have ssh access to the box and can set up stuff
15:13:42 <misa> repository admins have control over the repo's settings but not ssh access
15:13:56 <misa> an admin sets up the signing infrastructure and decides which keys are acceptable etc.
15:13:59 <dkliban> rest api access
15:14:09 <misa> dkliban: yep
15:14:28 <bmbouter> that works for me
15:14:51 <dkliban> the repository admin needs to be able to configure a 'signing service' and then add that service to a repository
15:15:04 <x9c4> So the admin needs to put the private key into place, and pulp mus know which repositoy admin may use whicgh one.
15:15:06 <misa> repo admins can set up a subset of the keys the admin has configured
15:15:32 <dkliban> so should the repository admin be able to upload a private key to Pulp?
15:15:38 <bmbouter> pulp_ansible will probably want to accept these keys over the API
15:15:49 <bmbouter> the people configuring the system and those who have the keys are different
15:15:50 <fao89> Triage will start in 15 minutes!
15:16:17 <bmbouter> but one of these 'signing service' could be self-manage
15:16:23 <misa> dkliban: it seems highly insecure, but if they so choose, maybe
15:16:36 <daviddavis> fao89: let's start triage after this signing discussion wraps up (may not be in 15 min)
15:16:49 <x9c4> So signing-services is a list in the global config?
15:17:03 <dkliban> daviddavis: we will stop at 10:30 ... i will schedule a follow up discussion
15:17:09 <daviddavis> ok
15:17:24 <misa> dkliban: if there were a pulp_gpg plugin, it would allow you to manipulate keys, but an admin may choose not to use it, or to use it in conjunction with an external service
15:17:44 <iballou> hey, I noticed that sync with mirror=false is no longer additive, I added a bug here https://pulp.plan.io/issues/5809
15:17:59 <daviddavis> iballou: great, we'll triage it today
15:18:00 <dkliban> iballou: cool. we are in a meeting now. thank you!
15:18:15 <bmbouter> we should get a meeting channel at some point
15:18:26 <x9c4> At pulpcon we kind of agreed, that such a plugin would only manage public keys.
15:18:53 <misa> and if the repo admin chooses to use the pulp_gpg infrastructure, then they implicitly can manage private keys
15:18:55 <bmbouter> x9c4: yes but we have new info
15:19:03 <misa> x9c4: I think managing private keys is a terrible idea
15:19:31 <bmbouter> yes but you also control the external signing service and pulp too right?
15:19:38 <misa> x9c4: but if they want to walk for 10 steps, who are we to tell them the end of the cliff is at 5 steps...
15:20:52 <daviddavis> lol
15:20:58 <x9c4> bmbouter, but you don't neccessarily controll them via REST and HTTP, but ssh and bash...
15:21:02 <x9c4> misa++
15:21:03 <pulpbot> x9c4: misa's karma is now 22
15:21:11 <dkliban> so what i am hearing so far is that we want to provide a  REST API that allows a user to configure a 'SigningService'
15:21:14 <misa> x9c4: maybe in a dev/test/prod scenario, you can accept devs to sign their packages using pulp, but in prod they always have to be signed by the external service
15:21:47 <misa> low-grade vs. high-grade security levels etc
15:22:16 <x9c4> So you just don't put the proper keys in the wrong places.
15:22:32 <misa> I think so.
15:22:46 <misa> with an external signing service, you typically don't even have access to the private key
15:22:47 <dkliban> THen we want to be able to associate that SigningService with a repository
15:22:51 <x9c4> Then one of the signing services could use those keys.
15:22:56 <misa> it's baked in some kind of a hardware appliance
15:23:26 <misa> dkliban: probably plural
15:23:44 <misa> dkliban: admin can set up one or more signing services, one of them being provided by pulp_gpg (in the future)
15:24:08 <misa> dkliban: and indeed a repository would be associated with a specific signing service
15:24:25 <misa> a signing service can be as simple as a config file like the one in https://github.com/pulp/pulp_deb/blob/2-master/README.md#signing-support
15:25:02 <dkliban> misa: yes, we will store the ocnfig in the DB though
15:25:38 <misa> dkliban: sure, although hopefully there's no REST API for it
15:25:39 <dkliban> misa: there will be different types of SigningService. and they will each provide a common interface for the plugin writer to create a signed artifact
15:26:38 <misa> I think the instance-level config should be command-line only for now
15:26:52 <misa> you can certainly add REST APIs for it later
15:27:18 <dkliban> we are almost at the end of our hour. i will send out a summary of this discussion to the ist. including the outstanding questions about the REST API for ocnfiguring signing services.
15:27:20 <misa> but that would become complicated really quickly
15:27:21 <bmbouter> so pulp_ansible would work very differently than this
15:27:28 <dkliban> i will then schedule another discussion for next Thursday
15:27:48 <misa> bmbouter: I have no experience with it, so please share
15:27:48 <bmbouter> so maybe we should consider implementing a few in the plugins and seeing where the common parts are
15:27:57 <bmbouter> I can explain some next time
15:28:02 <misa> +1
15:28:15 <dkliban> thank you everyone for participating today!
15:28:20 <x9c4> +1
15:28:22 <bmbouter> misa: I can see how regardless of pulp_ansible what you are describing is very valuable
15:28:27 <ttereshc> dkliban, did you mean Tuesday?
15:28:28 <bmbouter> +1
15:28:44 <bmbouter> I grabbed the tues slot I think
15:28:48 <bmbouter> for RBAC
15:28:54 <ttereshc> ah
15:28:55 <ttereshc> ok
15:28:58 <dkliban> #endmeeting
15:28:58 <dkliban> !end