14:30:46 <asmacdo> #startmeeting Pulp Triage 2016-07-01
14:30:46 <asmacdo> #info asmacdo has joined triage
14:30:47 <pulpbot> Meeting started Fri Jul  1 14:30:46 2016 UTC and is due to finish in 60 minutes.  The chair is asmacdo. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:30:47 <pulpbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:30:47 <pulpbot> The meeting name has been set to 'pulp_triage_2016_07_01'
14:30:47 <pulpbot> asmacdo has joined triage
14:30:48 <bmbouter> feel free to ping me in #pulp
14:30:49 <bmbouter> @here
14:30:51 <bmbouter> !here
14:30:51 <bmbouter> #info bmbouter has joined triage
14:30:52 <pulpbot> bmbouter has joined triage
14:30:57 <preethi> !here
14:30:57 <preethi> #info preethi has joined triage
14:30:58 <pulpbot> preethi has joined triage
14:31:34 <dalley> !here
14:31:34 <dalley> #info dalley has joined triage
14:31:34 <pulpbot> dalley has joined triage
14:31:34 <asmacdo> https://pulp.plan.io/issues?query_id=30
14:31:36 <pulpbot> Title: Un-Triaged Bugs - Pulp (at pulp.plan.io)
14:31:39 <dkliban> !here
14:31:39 <dkliban> #info dkliban has joined triage
14:31:39 <pulpbot> dkliban has joined triage
14:31:45 <asmacdo> !next
14:31:46 <pulpbot> 7 issues left to triage: 2039, 2042, 2045, 2047, 2049, 2050, 2051
14:31:46 <asmacdo> #topic Not full importer config is validated in the update call - http://pulp.plan.io/issues/2039
14:31:47 <pulpbot> Pulp Issue #2039 [POST] (ipanova@redhat.com) - Priority: Normal | Severity: Medium
14:31:47 <pulpbot> Not full importer config is validated in the update call - http://pulp.plan.io/issues/2039
14:31:55 <pcreech> !here
14:31:55 <pcreech> #info pcreech has joined triage
14:31:56 <pulpbot> pcreech has joined triage
14:32:10 <asmacdo> already in post
14:32:14 <asmacdo> !propose accept
14:32:14 <asmacdo> #idea Proposed for #2039: Leave the issue as-is, accepting its current state.
14:32:14 <pulpbot> Proposed for #2039: Leave the issue as-is, accepting its current state.
14:32:26 <pcreech> +1
14:32:34 <ttereshc> it is in post but Ina told me that she want the team to be aware of this bug
14:32:45 <asmacdo> ah ok
14:32:45 <ttereshc> I do not know much more about it
14:32:58 <mhrivnak> !here
14:32:58 <mhrivnak> #info mhrivnak has joined triage
14:32:58 <pulpbot> mhrivnak has joined triage
14:33:02 <ttereshc> but that is the reason why it is not marked as triaged
14:33:28 <mhrivnak> +1 accept
14:33:34 <asmacdo> anyone else?
14:33:55 <asmacdo> !accept
14:33:55 <asmacdo> #agreed Leave the issue as-is, accepting its current state.
14:33:56 <pulpbot> Current proposal accepted: Leave the issue as-is, accepting its current state.
14:33:57 <asmacdo> #topic updateinfo.xml is missing checksums - http://pulp.plan.io/issues/2042
14:33:57 <pulpbot> 6 issues left to triage: 2042, 2045, 2047, 2049, 2050, 2051
14:33:57 <pulpbot> RPM Support Issue #2042 [NEW] (unassigned) - Priority: Normal | Severity: Medium
14:33:58 <pulpbot> updateinfo.xml is missing checksums - http://pulp.plan.io/issues/2042
14:34:43 <mhrivnak> ttereshc, so am I reading this correctly that the default behavior changed with 2.9 regarding how checksums are included?
14:35:20 <ttereshc> before 2.9 pulp did not handle multiple checksums atll
14:36:08 <ttereshc> oh, that's another issue, sorry
14:37:03 <ttereshc> yes we publish the checksum of the distributor checksum_type for the packages in the erratum
14:37:13 <mhrivnak> I think what ichimonji10 is pointing out with this bug is that if you just sync and publish a repo with default settings, in 2.8 you see checksums in errata, and in 2.9 you don't.
14:37:14 <jcline> !here
14:37:14 <jcline> #info jcline has joined triage
14:37:15 <pulpbot> jcline has joined triage
14:37:30 <mhrivnak> I assume you don't because the checksums don't match the checksum used for the rest of the repo metadata?
14:38:35 <mhrivnak> If that's true, perhaps we should relax that and allow the previous behavior to continue. So by default it will publish whatever checksum value is available, and will only change behavior if the new setting is used.
14:39:00 <mhrivnak> Sorry to get into the weeds on details, but I'm trying to confirm whether we think this is a bug. :)
14:39:13 <ichimonji10> mhrivnak: That's exactly it.
14:39:21 <asmacdo> im afraid i am not familiar enough with this one to add to the conversation, perhaps lets skip this and discuss on comments
14:39:24 <ttereshc> if there are multiple checksums and none is matched the repo checksum, what to do?
14:39:48 <ttereshc> in 2.8 it would be an error
14:40:02 <ichimonji10> ttereshc: The repo distributor has *no* checksum configured
14:40:22 <mhrivnak> Hm, ok. I'm with asmacdo that we can continue this discussion post-triage and move on for now. ttereshc ichimonji10 does that work for you?
14:40:23 <ttereshc> ichimonji10, then it is sha256, iirc
14:40:43 <ttereshc> mhrivnak, sure
14:40:48 <ichimonji10> ttereshc: Let's catch up on this via video chat later
14:40:50 <asmacdo> !propose skip
14:40:50 <asmacdo> #idea Proposed for #2042: Skip this issue for this triage session.
14:40:50 <pulpbot> Proposed for #2042: Skip this issue for this triage session.
14:40:51 <ichimonji10> It's an interesting one
14:41:07 <asmacdo> ichimonji10, mhrivnak ttereshc lets try to do that soon
14:41:19 <asmacdo> if this is a bug, it sounds like it might need to block a release?
14:41:23 <ichimonji10> I'm free the rest of today
14:41:31 <ichimonji10> *most of today
14:41:33 <asmacdo> !accept
14:41:33 <asmacdo> #agreed Skip this issue for this triage session.
14:41:33 <pulpbot> Current proposal accepted: Skip this issue for this triage session.
14:41:35 <asmacdo> #topic Task stuck at waiting if child process segfaults - http://pulp.plan.io/issues/2045
14:41:35 <pulpbot> 5 issues left to triage: 2045, 2047, 2049, 2050, 2051
14:41:35 <pulpbot> Pulp Issue #2045 [NEW] (unassigned) - Priority: Normal | Severity: High
14:41:36 <pulpbot> Task stuck at waiting if child process segfaults - http://pulp.plan.io/issues/2045
14:41:48 <bmbouter> jortel and I discovered this together
14:42:02 <asmacdo> oof.
14:42:06 <bmbouter> I wanted to triage it onto the sprint acually
14:42:17 <bmbouter> or triage it not on the sprint but to high/high
14:42:22 <jcline> This sounds similar to a bug rbarlow filed about tasks that got OOM-killed also waited forever
14:42:36 <bmbouter> jcline: oh yes I do remember something about that
14:42:47 <bmbouter> let's triage this one and I can search to deduplicate
14:42:49 <mhrivnak> !propose triage high high
14:42:49 <mhrivnak> #idea Proposed for #2045: Priority: High, Severity: High
14:42:50 <pulpbot> Proposed for #2045: Priority: High, Severity: High
14:42:52 <jcline> https://pulp.plan.io/issues/1673
14:42:54 <pulpbot> Title: Issue #1673: Pulp's worker watcher does not notice workers that got killed by OOM killer and their tasks stay "running" forever - Pulp (at pulp.plan.io)
14:43:03 <mhrivnak> ah nice
14:43:06 <jcline> I wonder if they can both be fixed by one thing?
14:43:24 <mhrivnak> probably.
14:43:27 <bmbouter> they are almost certainly the same
14:43:32 <asmacdo> any thoughts re: put this on the sprint?
14:43:40 <bmbouter> the key detail that we know now is that it's specifically the child process
14:43:52 <bmbouter> the parent celery process does not have this problem
14:44:10 <bmbouter> oh there is some discussion of that on 1673 also actually
14:44:27 <bmbouter> ok so we can close the new one as a dup
14:44:30 <mhrivnak> Yep. It's a painful user experience when this happens, so it's a big opportunity for us to improve the experience.
14:44:46 <bmbouter> it will be easy I think, I outlined 3 solutions on 2045
14:44:53 <mhrivnak> bmbouter, perhaps let's discuss adding 1673 to a sprint separately?
14:45:01 <ttereshc> ichimonji10, mhrivnak, asmacdo, I can talk after triage
14:45:10 <asmacdo> thanks ttereshc
14:45:10 <bmbouter> sounds good, let's close 2045 as a dup then
14:45:23 <jluza> bmbouter, celery doesn't have watch dog for tasks?
14:45:30 <asmacdo> !propose other close as suplicate of #1673
14:45:30 <asmacdo> #idea Proposed for #2045: close as suplicate of #1673
14:45:30 <pulpbot> Proposed for #2045: close as suplicate of #1673
14:45:33 <asmacdo> !propose other close as duplicate of #1673
14:45:33 <asmacdo> #idea Proposed for #2045: close as duplicate of #1673
14:45:34 <pulpbot> Proposed for #2045: close as duplicate of #1673
14:45:36 <bmbouter> it does the defect is in the pulp bookeeping
14:46:00 <jluza> bmbouter, ah
14:46:02 <bmbouter> asmacdo: I'm going to go ahead and close it now as a dup
14:46:14 <asmacdo> bmbouter++
14:46:14 <pulpbot> bmbouter's karma is now 22
14:46:53 <asmacdo> !accept
14:46:53 <asmacdo> #agreed close as duplicate of #1673
14:46:54 <pulpbot> Current proposal accepted: close as duplicate of #1673
14:46:55 <asmacdo> #topic directory creation race condition during publish - http://pulp.plan.io/issues/2047
14:46:55 <pulpbot> 4 issues left to triage: 2047, 2049, 2050, 2051
14:46:55 <pulpbot> Pulp Issue #2047 [NEW] (unassigned) - Priority: Normal | Severity: Medium
14:46:56 <pulpbot> directory creation race condition during publish - http://pulp.plan.io/issues/2047
14:47:08 * mhrivnak looks for a katello person
14:47:14 <mhrivnak> partha !
14:47:35 <mhrivnak> maybe too early :)
14:47:50 <mhrivnak> Sat QE hit this problem.
14:48:16 <mhrivnak> I'm a bit surprised, but we've seen race conditions like this before that seemed unlikely, but they find a way to run into them.
14:48:43 <bmbouter> yeah I read this issue and it made sense to me
14:48:46 <bmbouter> as did the recommended solution
14:48:55 <asmacdo> more pythonic anyway
14:49:03 <asmacdo> !propose accept
14:49:03 <asmacdo> #idea Proposed for #2047: Leave the issue as-is, accepting its current state.
14:49:04 <pulpbot> Proposed for #2047: Leave the issue as-is, accepting its current state.
14:49:08 <mhrivnak> +1
14:49:54 <asmacdo> !accept
14:49:54 <asmacdo> #agreed Leave the issue as-is, accepting its current state.
14:49:54 <pulpbot> Current proposal accepted: Leave the issue as-is, accepting its current state.
14:49:56 <asmacdo> #topic Django RemovedInDjango110Warning in logs for missing TEMPLATES setting - http://pulp.plan.io/issues/2049
14:49:56 <pulpbot> 3 issues left to triage: 2049, 2050, 2051
14:49:56 <pulpbot> Pulp Issue #2049 [NEW] (unassigned) - Priority: High | Severity: Medium
14:49:57 <pulpbot> Django RemovedInDjango110Warning in logs for missing TEMPLATES setting - http://pulp.plan.io/issues/2049
14:50:50 <asmacdo> bmbouter, is this a dupe?
14:50:53 <jcline> No
14:51:00 <jcline> Well
14:51:07 <mhrivnak> Do we use django's template loading feature?
14:51:13 <jcline> There is a second one for a different deprecation
14:51:16 <jcline> Yes we do
14:51:30 <mhrivnak> just curious: do know where or what for?
14:52:27 <jcline> Rendering human-readable HTTP 404 and HTTP 500s, specifically in the content wsgi app
14:52:38 <bmbouter> oh that could be
14:52:42 <bmbouter> because I don't see it here:  https://github.com/pulp/pulp/blob/master/server/pulp/server/webservices/settings.py#L21-L26
14:52:43 <pulpbot> Title: pulp/settings.py at master · pulp/pulp · GitHub (at github.com)
14:52:45 <mhrivnak> gotcha.
14:52:46 <bmbouter> but that is different WSGI ap
14:52:47 <bmbouter> app
14:53:20 <mhrivnak> FWIW I actually made a pass at adding this to settings.py recently, but abandoned it because compatibility with django 1.4 was challenging.
14:53:37 <jcline> Well we're about to be in a pickle, then
14:53:41 <bmbouter> yes we are
14:53:43 <mhrivnak> Maybe not impossible, but it was at least annoying, and I determined I could continue without it.
14:53:48 * jcline chants SCL
14:54:05 * mhrivnak chants EL7 ;)
14:54:11 <bmbouter> mhrivnak: oh yeah I remember that
14:54:31 <mhrivnak> !propose accept
14:54:31 <mhrivnak> #idea Proposed for #2049: Leave the issue as-is, accepting its current state.
14:54:31 <pulpbot> Proposed for #2049: Leave the issue as-is, accepting its current state.
14:54:37 <jcline> We can always do a version check, it shouldn't be difficult
14:54:45 <mhrivnak> one way or another, we need to get to the bottom of this.
14:55:11 <asmacdo> any opposed to accept?
14:55:15 <bmbouter> jcline: I was thinking verion check too
14:55:16 <bmbouter> +1 to accept
14:55:20 <bmbouter> good file jcline
14:55:32 <asmacdo> !accept
14:55:32 <pulpbot> Current proposal accepted: Leave the issue as-is, accepting its current state.
14:55:32 <asmacdo> #agreed Leave the issue as-is, accepting its current state.
14:55:34 <pulpbot> 2 issues left to triage: 2050, 2051
14:55:35 <asmacdo> #topic HTTP 500 rather than a HTTP 404 when a missing resource is requested - http://pulp.plan.io/issues/2050
14:55:35 <pulpbot> Pulp Issue #2050 [NEW] (unassigned) - Priority: High | Severity: Low
14:55:35 <pulpbot> HTTP 500 rather than a HTTP 404 when a missing resource is requested - http://pulp.plan.io/issues/2050
14:56:12 <bmbouter> this issue is written generally, but it's specific to that 1 URL
14:56:29 <bmbouter> oh no wait it is for any URL that doesn't match
14:56:30 <jcline> bmbouter, are you certain of that?
14:56:59 <mhrivnak> !propose accept
14:56:59 <mhrivnak> #idea Proposed for #2050: Leave the issue as-is, accepting its current state.
14:56:59 <pulpbot> Proposed for #2050: Leave the issue as-is, accepting its current state.
14:57:09 <mhrivnak> definitely bad.
14:57:59 <bmbouter> jcline: yeah it stays in all django code so this is a general problem
14:58:20 <asmacdo> other thoughts?
14:58:43 <jcline> I'm fine with accepting it as-is, naturally
14:58:56 <asmacdo> !accept
14:58:56 <asmacdo> #agreed Leave the issue as-is, accepting its current state.
14:58:56 <pulpbot> Current proposal accepted: Leave the issue as-is, accepting its current state.
14:58:57 <pulpbot> 1 issues left to triage: 2051
14:58:58 <asmacdo> #topic Ludicrous number of WARNING logs from publish - http://pulp.plan.io/issues/2051
14:58:58 <pulpbot> RPM Support Issue #2051 [NEW] (unassigned) - Priority: High | Severity: High
14:58:58 <pulpbot> Ludicrous number of WARNING logs from publish - http://pulp.plan.io/issues/2051
14:59:43 <mhrivnak> are the warnings about something specific?
15:00:30 <jcline> mhrivnak, see the log snippet in the report. That's all there is.
15:00:31 <bmbouter> it looks like NEVRA
15:00:56 <mhrivnak> Yep, it's just hard to read and appears truncated.
15:01:21 <mhrivnak> In any case, it's highly unfortunate and we should fix it. :)
15:01:23 <bmbouter> it's likely one line for all the statements in a loop
15:01:24 <jcline> I didn't think anyone wanted to read 11k NEVRAs
15:01:41 <asmacdo> do we want to propose just lowering to DEBUG?
15:01:44 <asmacdo> and adding easyfix?
15:01:52 <bmbouter> both of those sound good
15:01:55 <jcline> The logs have a max line length and will get truncated
15:02:21 <asmacdo> !propose triage high low
15:02:21 <asmacdo> #idea Proposed for #2051: Priority: High, Severity: Low
15:02:21 <pulpbot> Proposed for #2051: Priority: High, Severity: Low
15:02:29 <bmbouter> let's not propose DEBUG change and just triage with adding easy fix
15:02:49 <asmacdo> bmbouter, i meant to add DEBUG as a comment
15:03:01 <mhrivnak> Oh ya, it's just one line.
15:03:04 <mhrivnak> easy fix.
15:03:22 <asmacdo> thats ok though, im sure whoever picks this up will evaluate what to do
15:03:46 <asmacdo> i changed severity from high to low, is everyon ok with that?
15:03:48 <asmacdo> jcline, ?
15:03:50 <mhrivnak> +1
15:04:09 <bmbouter> it does make the logs unusable
15:04:12 <jcline> I don't really mind one way or the other, but it's very ruinous to the logs
15:04:37 <bmbouter> high/low sounds ok since the prio is high
15:04:51 <asmacdo> k
15:05:00 <asmacdo> #agreed Priority: High, Severity: Low
15:05:00 <asmacdo> !accept
15:05:00 <pulpbot> Current proposal accepted: Priority: High, Severity: Low
15:05:01 <pulpbot> No issues to triage.
15:05:07 <bmbouter> yay
15:05:18 <asmacdo> !end
15:05:18 <asmacdo> #endmeeting