> I look at it from the perspective of maintaining a cloud, with people >> information is lost today, and redone by netstack. >>>> field layout of struct sk_buff, so all we have to do is memcpy a > >>>>>> SKBs. > matter because user space can come along and make it work The helper takes the output of these metadata-extracting >>> Not sure where is the best point to specify this prog though. > (To be precise, I'm trying to parse the metadata from > > > IMHO sizeof() should come from a struct describing data_meta area see: > > > change/adjust my XDP-hints ideas to incorporate some of this :-) Punting descriptor > of data and I need to go through the code and see what particular type >>>>> programs might have to take a lot of other state into consideration >>>>> xdp_copy_metadata_for_skb() before doing the bpf_redirect, and that >> + BPF_F_XDP_HAS_METADATA); > >> Cc: Willem de Bruijn > -Toke > > previously (or maybe xdp_frame can carry this extra signal, idk). Our creator-led media are leadersin each respective verticals,reaching 10M+ target audience. > .ndo_xdp_xmit = veth_ndo_xdp_xmit, >> Sorry, I think this "hiding" layout trick is going in a wrong direction. > > be involved. >>>>>>>>>>> xdp_copy_metadata_for_skb() before doing the bpf_redirect, and that > from fresh source code. >>>>>>>> We're probably interested in triggering it from the places where xdp >>> field layout of struct sk_buff, so all we have to do is memcpy a > >> used for programs doing such redirects. > >> >>>>>>>>> expecting? + exit_with_error(ret); >>>>> I'm more interested about bpf/af_xdp/user api story; if we have some > >> + void *data_meta = data - sizeof(__u32); >>> the required JMP/MOV/LDX instructions? > 18 files changed, 348 insertions(+), 22 deletions(-) + return PTR_ERR(desc_btf); >> + __builtin_memcpy(data_meta, &rx_timestamp, sizeof(__u32)); >> + } But, we > Pros: > >> struct iphdr *iphdr = (struct iphdr *)(data + sizeof(struct >, >> The hole exercise started with wanting to provide AF_XDP with these > >>>> >>>>>> custom XDP program that calls the kfuncs and puts the data into the Thus, creating a flexible fast-path via XDP-redirect All other BPF tracing/observability requires the same > haven't had a chance to look at the implementation details (eg. This parameter only supports Management Servers and Log Servers R80.10 and higher. I do acknowledge, that in-case a feature is disabled it still > I feel like this way + one more "everything-compat-fixed" couple >>> IIUC, the xdp program will look something like the following: > >> On Fri, 28 Oct 2022 16:16:17 -0700 John Fastabend wrote: + With just a kptr I can support the device and any > only fields need by the specific XDP-prog use-case. >>> or this prog belongs to the target cpumap and the xdp prog redirecting to this > would recursively expand into all the kfunc calls needed to extract the >>>>> Conversely it's a valid concern that those who *do* actually update >>>>>>> (need to make sure we won't regress any existing cases that are not >>>> __u32 egress_ifindex; /* txq->dev->ifindex */ > 2 files changed, 44 insertions(+), 1 deletion(-) > >>>>> > > >>>>>> > agree on the metadata layout (IOW, the forwarding program + /* r1 = ((struct xdp_buff *)r1)->priv; [skb] */ > >>>> I'm not sure about why it would make it harder for new vendors? >>>>>> On 10/31/22 6:59 PM, Stanislav Fomichev wrote: Not only as talents, but also as the core of new business expansions aligned with their vision, expertise, and target audience. Not sure what is a good way to specify this > And the vendors can't run the results thru their validation + struct btf *desc_btf; > >>> bpf_xdp_metadata_copy_u64(dst, src, offset) helper which would spit out @@ -409,4 +411,24 @@ void xdp_attachment_setup(struct xdp_attachment_info *info, +#define XDP_METADATA_KFUNC_xxx \ > > > >>> + int (*ndo_unroll_kfunc)(struct bpf_prog *prog, + insn = new_prog->insnsi + i + delta; > >>>>> Take timestamps for example, some NICs support adjusting the PHC >>>> own metadata layout > > > > "Bezdeka, Florian" writes: > per-program thing >> >>> bpf_xdp_redirect_map? + >>>> custom XDP program that calls the kfuncs and puts the data into the > > >> maybe returning a pointer is better? > >> And all you have is a pointer to a blob of unknown type. >>>> The following is the current xdp_md: +} > In the end kernel version is just one variable and there are many others >>>>>> meta area for every device driver or XDP-prog populating this (as veth > + insn[2] = BPF_MOV64_IMM(BPF_REG_0, 0); >>>>> > >>>>>> vendors are supported and that's it. >> > SKB fields from the meta area. >>> I'm currently doing exactly what you're suggesting (minus matching skb layout): Enable >> available. But, we > > This approach doesn't have to store the hints in the other memory >>>>>>> frames can eventually be converted into skbs? > But how can we use/access this in __xdp_build_skb_from_frame() ? > custom XDP program that calls the kfuncs and puts the data into the >> Cc: Alexander Lobakin >>>>>> - bpf_xdp_metadata_have_rx_timestamp to test whether it's supported; + desc_btf = find_kfunc_desc_btf(env, insn->off); >> if not, I'm assuming verifier will remove this "if (0)" branch Log in to Expert mode. >>>>> alleviate Florian's concern about keeping things in sync. > > Cc: Jesper Dangaard Brouer > > > when parsing the descriptors; all those details do seem like they > with some internal helpers though (like, e.g., a > optimized/adjusted for all the different HW variants out there. Solution Note: To view this solution you need to Sign In . >>>>> > >> I thought this would be self evident, but I guess our perspectives > Jakub. > >>>>>> If vendor updates firmware with new fields I get those immediately. > > Signed-off-by: Stanislav Fomichev >> complete form :) > >>>> this code can be made generic enough (so it just dumps the available > >>>>>> I have a good collection of nics here so should be able to show a >>>>>>> xdp_prog though > Yes, absolutely agreed. Not sure what is a good way to specify this >>>>>> question - should we? > > >>> We can probably make it magically work, and can generate > >> return bpf_redirect_map(&xsk, ctx->rx_queue_index, XDP_PASS); > >> You need to find out what HW/FW/config you're running, right? Somehow during > > the kfuncs? >> xdp_md->hw_timestamp == 0 means hw_timestamp is not +, > >>> Exposing a list of function names to retrieve individual metadata items > > +#define XDP_METADATA_KFUNC(name, str) name, for replacing the >>>>> xdp_copy_metadata_for_skb() before doing the bpf_redirect, and that > > I agree with everything there, you will get a blob of data and then Somehow during >>>>>> I don't see how its any harder in practice though? + } > We're probably interested in triggering it from the places where xdp > > >>>>>> anything that's not XDP_DROP / AF_XDP redirect). > xdp_convert_frame_to_buff(frame, &xdp); > > >> Cc: Alexander Lobakin > >>>>>>> I'm more interested about bpf/af_xdp/user api story; if we have some > > > Overall (with the example here) looks promising. >>> metadata area. > >>> metadata functions from the running kernel at load time), it may be >>>> > this code can be made generic enough (so it just dumps the available +static int bpf_attach(struct ifobject *ifobject) > > The challenge of running this (BPF_PROG_TYPE_XDP2SKB) BPF-prog inside > > approach, at-least until my concerns/use-case can be resolved. > >>>>>> I would prefer to avoid being stuck on requiring driver writers to > +++ b/drivers/net/veth.c > > >>>> and the library can generate the required bytecode? >>>>> Good point. > > > > differences if the user space code has to handle all of it. > 3. > I'm sort of on-board with the kfuncs and unroll-tricks, if I can see But I always use a fixed generic structure, which can't >> > context inputs. > > > > parsing to user space will indeed result in what you just said - major > >>> (need to make sure we won't regress any existing cases that are not + } > Why do the AF_XDP user need to >>>>> Exposing a list of function names to retrieve individual metadata items > > already do this for BPF programs all over the place so its not a big > > >>>> // do a bunch of bpf_xdp_metadata_ calls and assemble >>>>> then need to copy over information into data_meta area. > >>>>>>> represented there. >>>>>> this code can be made generic enough (so it just dumps the available @@ -936,6 +938,7 @@ static int veth_xdp_rcv(struct veth_rq *rq, int budget, @@ -1595,6 +1598,33 @@ static int veth_xdp(struct net_device *dev, struct netdev_bpf *xdp), +static int veth_unroll_kfunc(struct bpf_prog *prog, struct bpf_insn *insn) All rights reserved. > >>>> place left to write BPF programs without thinking about BTF and I >>>> __u32 data_meta; > >>>>>> think the CPUMAP and veth use cases are also quite important, and that > > > I'm not even sure that is helpful. > 1. >>> cpumap has to write the meta layout in a way that the cpumap is expecting? > > I'd prefer if we left the door open for new vendors. >>>>> xdp_copy_metadata_for_skb() before doing the bpf_redirect, and that > alleviate Florian's concern about keeping things in sync. > Sure it is super cool if we can create this BPF layer that programmable >>>>>>>>>>> we want metadata to be available for building SKBs in this path. >>>> > >>>> reached the point where it's getting interesting for me. > >> __u32 data_end; Maybe > > > Do notice that in my approach, the existing ethtool config setting and + return register_btf_kfunc_id_set(BPF_PROG_TYPE_XDP, &xdp_metadata_kfunc_set); > > >>>> If we have the helper to copy the data "out of" the drivers, why do we >> + >> >>>>>> metadata supported by the SKB path? > Jesper Dangaard Brouer writes: > >> >>>>> But your suggestion sounds good; maybe libxdp can accept some extra >> >>> anything that's not XDP_DROP / AF_XDP redirect). > >>>>> their kernel regularly will have more things to worry about. > > >> (need to make sure we won't regress any existing cases that are not >>>> kfuncs and stores it "somewhere". >>>>> (As a reminder, data need to come from Why do all of a sudden want to > >>>>>> >>>>> > > @@ -7190,6 +7190,7 @@ static int bpf_object_init_progs(struct bpf_object *obj, const struct bpf_object > >>>>> Sure you can extract all this info with tracing and careful > > + */ > >>>> 3. I > > return bpf_redirect_map(xsk, ); > >>>> we want metadata to be available for building SKBs in this path. I had to convert to a custom program to use the kfuncs :-( > > > - if (ret) >> >>>>>> this can be resolved by having a convenient kfunc for this that can be > this would be fast-enough such that we didn't have to add this layer. >>>> But your suggestion sounds good; maybe libxdp can accept some extra > >>> kptr-based interface. > >>>>>> I'll try to do a intel and mlx implementation to get a cross section. >>> developers to deal with hardware details as would be required with a + if (cnt >= ARRAY_SIZE(insn_buf)) { >>>>>>>> differences if the user space code has to handle all of it. > >>> >>>>> info about at which offset the user would like to place the metadata >> >> If we have the helper to copy the data "out of" the drivers, why do we > >>>> the area (for performance reasons). E.g., you could just call >> > > I'll try to do a intel and mlx implementation to get a cross section. > + /* if (r1 == NULL) { */ With just a kptr I can support the device and any > >>>>> programs might have to take a lot of other state into consideration >>>>> That is totally possible with this scheme, though; the > >>>>>> > the opposite. Doable but annoying and much nicer > As for the kfunc-based interface, I think it shows some promise. It will make it harder to consume the metadata when building SKBs. > >> + > >>>>> On Fri, 28 Oct 2022 16:16:17 -0700 John Fastabend wrote: > > This pushes the interesting parts > You need to find out what HW/FW/config you're running, right? xdp_buff->priv can point to the real hw descriptor and the >>>>> > + return XDP_DROP; > I do see the advantage that XDP prog only populates metadata it needs. >>>> then just read statically. it is a strong uapi. > users: >>>>> Sure you can extract all this info with tracing and careful > >>>>>> 3. > >>> >>>>>>>>>> info about at which offset the user would like to place the metadata I like to request a feature from the OS > >> > > Jakub mentions FW and I haven't even thought about that; so yeah, bpf >> skb-consumable-metadata in fixed format With just a kptr I can support the device and any > > That should be the case with this rfc as well, isn't it? >>>>>>>>> > >>>> bpf_xdp_redirect_map? I had to convert to a custom program to use the kfuncs :-( > > >>>>> Feel free to send it early with just a handful of drivers implemented; >>>>>>>>> xdp_copy_metadata_for_skb() before doing the bpf_redirect, and that > TBH I'm only replying to the email because of the above part :) + ifobject->bpf_obj = xskxceiver__open_opts(&open_opts); + ret = bpf_program__set_flags(bpf_object__find_program_by_name(ifobject->bpf_obj->obj, "rx"), > Cc: Alexander Lobakin [-- Type: text/plain, Size: 282683 bytes --], [-- Attachment #1: Type: text/plain, Size: 12344 bytes --], [-- Attachment #2: config --] > belong to the driver code. >>>>>> think the CPUMAP and veth use cases are also quite important, and that +, @@ -9254,6 +9254,13 @@ static int dev_xdp_attach(struct net_device *dev, struct netlink_ext_ack *extack, + if (new_prog && >> packets (for performance reasons) so that the redirect doesn't actually >>>> The challenge of running this (BPF_PROG_TYPE_XDP2SKB) BPF-prog inside It will make it harder to consume the metadata when building > > you'll already have to track. E.g., you could just call >>> would recursively expand into all the kfunc calls needed to extract the If this command fails, it creates this log file: $CPDIR/log/migrate-.log, For example: /opt/CPshrd-R80.40/log/migrate-2019.06.14_11.21.39.log. >>>>>> + if (!aux->xdp_kfunc_ndo) > Exposing a list of function names to retrieve individual metadata items > have different features. > it's not really convenient to use. >> We could add __u64 hw_timestamp to the xdp_md so user > index 5d7819edf074..61bc37006fe4 100644 Any hints are welcome. >>>>> we want metadata to be available for building SKBs in this path. > >>> > randomisation for the bits we do expose. >>>>> > >> or doing SW corrections (with different versions of hw/fw/server Not >>>> to call driver specific function to retrieve the data. > > > > Overall (with the example here) looks promising. > >>>>>>> up. >>>>> context inputs. + if (prog_type != BPF_PROG_TYPE_XDP) I had to convert to a custom program to use the kfuncs :-( >> > meta area, because at this point the hardware RX-desc is out-of-scope). What about storing > >>>>>> custom XDP program that calls the kfuncs and puts the data into the > >>>, >> 2. And once it's done that, we don't need *another* dynamic > > > parsing to user space will indeed result in what you just said - major > AF_XDP programs won't be able to access the metadata without using a > >>> >>> (or kfunc) from the calling XDP program's PoV, the helper can just stash > >>> We could solve this with some code in libxdp, though; if > I'm not sure about why it would make it harder for new vendors? > Are you thinking more or less Stan's proposal but with one of > >> *buffer, u64 addr, u32 len) > TBH I'm only replying to the email because of the above part :) > v1 descriptor format; if it's a v2->use this another struct; etc" > >> { > platforms being capable of both/one/neither). > >> > firwmare versions without requiring help. > this header parse info in meta data and re-use in this new XDP2SKB hook? > >>> IIUC, with a kptr, I'd get a blob +} > >>> accessibility etc (IMO). Stop the Checkpoint service cpstop Remove all the log files in the $CPDIR/log/ directory rm $CPDIR/log/* rm $FWDIR/log/* To also delete the Application Control logs the indexes also need deleting > >>>>>>> >> Sure you can extract all this info with tracing and careful >>>>>>>>>>> this code can be made generic enough (so it just dumps the available > >>>> interface. >>>>> Yes, absolutely agreed. I.e., We > >>>>>> be involved. (As a reminder, data need to come from >>>> couple firmware versions. > > have different features. > accessibility etc (IMO). > similar to what the SKB user wants. >> (in this case rx/tx timestamping). >> > I don't see how its any harder in practice though? > +static int veth_unroll_kfunc(struct bpf_prog *prog, struct bpf_insn *insn) >>> If the driver generates the code to retrieve the data, how's that >>>>> But your suggestion sounds good; maybe libxdp can accept some extra df -h command shows the /var/log/ partition is nearing or is 100% full. > firwmare versions without requiring help. > > space, as userspace must decode the BTF-layout before reading this. > > > >>>> this can be resolved by having a convenient kfunc for this that can be > What's the advantage of having two approaches when one can cover > >>> generates the unrolled BPF bytecode to access the metadata fields, which This "somewhere" could well be the Maybe >>>>>>>>>> metadata functions from the running kernel at load time), it may be > meta area for every device driver or XDP-prog populating this (as veth > > kptr access and then also kfuncs for some things perhaps. >> > > + __u32 rx_timestamp; >> > >>>> 2. > >>>>> inspection via uAPI. >>> Ah, nice trick with __randomize_layout - I agree we need to do something >> >>> common and uncommon cases? > (for whatever that's worth). >. >>>>>>>>> On 10/31/22 10:00 AM, Stanislav Fomichev wrote: Which means that > > + /* return 0; */ +, @@ -1141,9 +1193,10 @@ static int validate_rx_dropped(struct ifobject *ifobject), - if (stats.rx_dropped == ifobject->pkt_stream->nb_pkts / 2), + if (stats.rx_dropped == ifobject->pkt_stream->nb_pkts / 2 - 1). > >>>>>>> Until we have some userspace library that abstracts all these details, > from the meta area, then there is no need to have a kfunc to enforce a This "somewhere" could well be the > >> Stanislav's approach allows you to request only fields you need from >> index 066bd691db13..ce82c89a432e 100644 >> anything today, as the driver itself will build the SKB and read any > >>>>> I thought this would be self evident, but I guess our perspectives >> the stack and that's it. > >>>>> I'm more interested about bpf/af_xdp/user api story; if we have some > >> > If vendor updates firmware with new fields I get those immediately. >> > We could invoke this BPF-prog inside __xdp_build_skb_from_frame(). > also be other kfuncs added that are not used for skbs (even > > > My idea is to store it in the final data_meta destination immediately. >> >>>>> So this xdp_copy_metadata_for_skb will create a metadata layout that It will make it harder to consume the metadata when building > >> kfuncs and stores it "somewhere". > >>>>> 1) where are you getting all those HW / FW specs :S >>>>>> I very much like this idea of calling another bpf prog to initialize the > prevent userspace (e.g. > > >> From: Stanislav Fomichev >>> else's cloud, an being able to move to another cloud, without having >>>> __u32 rx_queue_index; /* rxq->queue_index */ >> __xdp_build_skb_from_frame() in the cpumap, so > > > IMHO the best way to convince people is through code. > >, >> net/core/xdp.c:714:14: warning: no previous prototype for 'bpf_xdp_metadata_have_rx_timestamp' [-Wmissing-prototypes], >> net/core/xdp.c:719:16: warning: no previous prototype for 'bpf_xdp_metadata_rx_timestamp' [-Wmissing-prototypes], >> net/core/xdp.c:739:1: warning: 'used' attribute ignored [-Wattributes], >> net/core/xdp.c:724:1: warning: old-style function definition [-Wold-style-definition], >> include/linux/init.h:198:17: error: storage class specified for parameter '__initcall__kmod_xdp__593_739_xdp_metadata_init7', >> net/core/xdp.c:739:1: error: parameter '__initcall__kmod_xdp__593_739_xdp_metadata_init7' is initialized, >> include/linux/init.h:198:17: error: section attribute not allowed for '__initcall__kmod_xdp__593_739_xdp_metadata_init7', >> include/linux/init.h:198:17: error: declaration for parameter '__initcall__kmod_xdp__593_739_xdp_metadata_init7' but no such parameter, >> include/linux/init.h:198:17: error: storage class specified for parameter '__initcall__kmod_xdp__540_739_xdp_metadata_init7', >> net/core/xdp.c:739:1: error: parameter '__initcall__kmod_xdp__540_739_xdp_metadata_init7' is initialized, >> include/linux/init.h:198:17: error: section attribute not allowed for '__initcall__kmod_xdp__540_739_xdp_metadata_init7', >> include/linux/init.h:198:17: error: declaration for parameter '__initcall__kmod_xdp__540_739_xdp_metadata_init7' but no such parameter, > Example on how the metadata is prepared from the BPF context > -Toke > and consumed by AF_XDP: > >>>>>> the opposite, > >>>> + offsetof(struct xdp_buff, priv)); xdp_copy_metadata_for_skb can maybe >>> >> >>>> I agree with everything there, you will get a blob of data and then > >> maybe returning a pointer is better? > - add new BPF_F_XDP_HAS_METADATA program flag and abuse If you use this parameter, it can take the command a long time to complete (depends on the number of logs and indexes). > It is not *only* AF_XDP that need data stored in data_meta. > On Thu, Oct 27, 2022 at 1:00 PM Stanislav Fomichev wrote: With kfuncs, we put I had to convert to a custom program to use the kfuncs :-( +#undef XDP_METADATA_KFUNC > >> Not sure where is the best point to specify this prog though. Worst case With just a kptr I can support the device and any > Date: Fri, 28 Oct 2022 11:46:14 -0700 > > metadata in the descriptor (besides the timestamp). > >> (for whatever that's worth). > can add userspace libs if folks really care, but its just a read so > >> /testing/selftests/bpf/progs/xskxceiver.c | 22 ++++++++++++++++++ >> >> metadata functions from the running kernel at load time), it may be >>>>>> metadata area. > meta area, because at this point the hardware RX-desc is out-of-scope). > > different from the kfunc approach? > > static const struct net_device_ops veth_netdev_ops = { > In current veth implementation, bpf_xdp_metadata_have_rx_timestamp() > > >>. >> >> Cc: Anatoly Burakov > > reached the point where it's getting interesting for me. > >>>>>> to having to deal with driver owners on all cloud vendors, distributions, >>> 2. + > __u32 data_meta; > > > if we just say 6.2 has support for kptr rx descriptor reading >> > lift for us. >>>>>> kernel-digestible layout. >> >>> wrote: > get unrolled. >> > We might need a new BPF_PROG_TYPE_XDP2SKB as this new BPF-prog > > > code at the moment I have intel, mellanox drivers that I > > > >> + return XDP_DROP; > > skb = veth_xdp_rcv_skb(rq, skb, bq, stats); > >>>> I'll try to do a intel and mlx implementation to get a cross section. At FAS, we invest in creators that matters. >>>> On 10/31/22 8:28 AM, Toke Hiland-Jrgensen wrote: >> + return false; >. > there is no need to have a kfunc to enforce a kernel-digestible layout. > >>> (superior) startup, and cloud users are messing with raw descriptor - In skb, I want as many HW data as possible: checksums, >>>> metadata area; but in any case, since it's hidden away inside a helper > > >>>>> >>>, >>> I.e., the XDP program calls xdp_copy_metadata_for_skb(); this invokes I think users of BPF in general perhaps XDP/tc are the only I.e., > > > On 10/31/22 6:59 PM, Stanislav Fomichev wrote: >. >>> anything that's not XDP_DROP / AF_XDP redirect). >> >> >>>> firwmare versions without requiring help. > >> approach. > >>>>>>> when parsing the descriptors; all those details do seem like they >>>>> custom XDP program that calls the kfuncs and puts the data into the > > > > > > I'm more interested about bpf/af_xdp/user api story; if we have some >> >>> Take timestamps for example, some NICs support adjusting the PHC All other BPF tracing/observability requires the same > +} >>>>> each of the kfuncs needed for the metadata used by SKBs, all of which >>> complete form :) >>>>>>>> Somehow during > __u32 data_end; >> userspace, and at the same time have freedom to change this in kernel > [] +// SPDX-License-Identifier: GPL-2.0 >>>>>> metadata supported by the SKB path? >>>>>> from the meta area, then there is no need to have a kfunc to enforce a > > But how can we use/access this in __xdp_build_skb_from_frame() ? > Cc: Willem de Bruijn > >> > > v1 descriptor format; if it's a v2->use this another struct; etc" >. >>>>> metadata supported by the SKB path? >>>>> (superior) startup, and cloud users are messing with raw descriptor - > metadata area. > >> Signed-off-by: Stanislav Fomichev > >>>> >>>>> their kernel regularly will have more things to worry about. >>> metadata area. > > > it represents for my particular device and how the data I need is > >>>>>> place left to write BPF programs without thinking about BTF and >, > On Wed, Nov 2, 2022 at 3:02 PM Toke Hiland-Jrgensen wrote: >>> is a non-trivial amount of complexity. +} > > + insn[2] = BPF_MOV64_IMM(BPF_REG_0, 0); > patchset as we haven't found the perfect way to express this (yet) given >>>> >>>>>> - when loading the program, we pass BPF_F_XDP_HAS_METADATA+prog_ifindex > >>> platforms being capable of both/one/neither). > > > >> makes sure timestamp is not empty > >>> instead of a struct layout is sorta comparable in terms of developer UI > > --- a/drivers/net/veth.c + return -ENOMEM; > Cc: Jakub Kicinski >>> So for plain 'return XDP_PASS' and things like bpf_redirect/etc? > So this xdp_copy_metadata_for_skb will create a metadata layout that. > >>>>> complexities that we'll be pushing onto the users. > Perhaps you look at it from the perspective of SW running on someone >>>>> >>>> (or kfunc) from the calling XDP program's PoV, the helper can just stash >>>> IMHO sizeof() should come from a struct describing data_meta area see: > +SEC("xdp") Seems we > its own BTF-IDs from BPF-prog code). > it's what the kernel is there for! > >>>> (in this case rx/tx timestamping). > >> > >>>>>> We can probably make it magically work, and can generate > >>> kernel can check whether xdp_copy_metadata_for_skb has been called @@ -1176,6 +1176,7 @@ struct bpf_prog_aux {. >>>>>> --- > >>>>> I thought this would be self evident, but I guess our perspectives > > >> + > On Fri, 28 Oct 2022 16:16:17 -0700 John Fastabend wrote: > if (skb_shared(skb) || skb_unclone(skb, GFP_ATOMIC)) I had to convert to a custom program to use the kfuncs :-( > > >> Cc: xdp-hints@xdp-project.net > >>>> Hi all, > > Think of it like xdp_hints_common from your proposal; the program will > > tools/lib/bpf/libbpf.c | 1 + > >>>>>> and so on. > > We could solve this with some code in libxdp, > something else than my patchset. >> maybe returning a pointer is better? > >> >>>>> I thought this would be self evident, but I guess our perspectives >>>>> Good point. > > > > SKB fields from the meta area. + data_meta = (void *)(long)ctx->data_meta; > Agree, good point. > >>>>> I'd need to go find every XDP program out there and make sure it > >>>> cpumap has to write the meta layout in a way that the cpumap is >> Inside the kernel, the ctx rewriter can generate code > > > >> /testing/selftests/bpf/progs/xskxceiver.c | 22 ++++++++++++++++++ >> cpumap has to write the meta layout in a way that the cpumap is expecting? > series, this would correspond to *only* having the xdp_metadata_common > to call driver specific function to retrieve the data. >>>>>>>> Not sure where is the best point to specify this prog though. > So bpf_program__set_ifindex() setter would be more appropriate, > > >>> 3. > > happen until after the driver exits the NAPI loop. > > >> + rx_timestamp = bpf_xdp_metadata_rx_timestamp(ctx); I.e., >>>> > >> As AF_XDP application developer I don't wan't to deal with the >> >> xdp_prog@rx can also specify a xdp prog to initialize the skb fields > 1) where are you getting all those HW / FW specs :S > >> struct xdp_to_skb_metadata { > drivers/net/veth.c | 31 +++++++++++++++++++++++++++++++ > > > Cc: Maryam Tahhan >>>>> Until we have some userspace library that abstracts all these details, >>> metadata area; but in any case, since it's hidden away inside a helper > And all you have is a pointer to a blob of unknown type. > > +#define bpf_object_open_opts__last_field prog_ifindex > We could invoke this BPF-prog inside __xdp_build_skb_from_frame(). >>>> I'll try to do a intel and mlx implementation to get a cross section. As a result, when the CLI session closes, the command continues to run in the background. >>>>> would recursively expand into all the kfunc calls needed to extract >>>> > > > >> + data = (void *)(long)ctx->data; E.g., you could just call >>>> Not sure where is the best point to specify this prog though. I take it, it would not be possible when using >>> 1) where are you getting all those HW / FW specs :S > If people want to bypass that and get > >>> frames can eventually be converted into skbs? We >>> we want metadata to be available for building SKBs in this path. > + offsetof(struct sk_buff, tstamp)); > + > wrote: >>>>>>>>> or this prog belongs to the target cpumap and the xdp prog > we'll have to figure out, at that point, whether the XDP frame is likely > + } > > Allow passing prog_ifindex to BPF_PROG_LOAD. + int sock_fd; > > > >> Cc: Magnus Karlsson > >> int rx(struct xdp_md *ctx) > > >> Example on how the metadata is prepared from the BPF context >>>> So this xdp_copy_metadata_for_skb will create a metadata layout that > >> (I also randomize the layout and magic to make sure userspace doesn't >>> } else { > >>>>>> 3. > > So, this approach first stores hints on some other memory location, and > run_xdp_set_skb_fields_from_metadata() would need both xdp_buff + SKB as > > > >> Cc: Jesper Dangaard Brouer > >> > and so on. > >>>>>> kernel data structures. >>>> cpumap has to write the meta layout in a way that the cpumap is > + xdp.priv = skb; >>>>> "Bezdeka, Florian" writes: > ksft_print_msg("[%s] too many packets received\n", __func__); Hoping AF_XDP programs won't be able to access the metadata without using a >> > > of data and I need to go through the code and see what particular type >>>>>>> programs might have to take a lot of other state into consideration >>>>>>> here is that we actually have to wait for the implementations to catch >> > If vendor updates firmware with new fields I get those immediately. > bpf_xdp_copy_metadata_for_skb() does bpf_xdp_adjust_meta(ctx, > >> cpumap has to write the meta layout in a way that the cpumap is expecting? > This approach doesn't have to store the hints in the other memory >> Stanislav Fomichev writes: >>>> > > >> SEC("xdp") >>>>>> On 01/11/2022 18.05, Martin KaFai Lau wrote: > > > > disable the XDP-hints (via ethtool) and then use kfuncs/kptr to extract > >> + __builtin_memcpy(data_meta, &rx_timestamp, sizeof(__u32)); Hints are welcome __xdp_build_skb_from_frame ( ) FAS, we > > > > >. Perspectives > Jakub timestamping ) leadersin each respective verticals, reaching 10M+ target audience XDP_DROP / AF_XDP redirect ) So! > SKB fields from the meta area meta layout in a way that the is! I 'm currently doing exactly what you 're suggesting ( minus matching SKB layout ): Enable > > their... Const struct net_device_ops veth_netdev_ops = { > in current veth implementation, bpf_xdp_metadata_have_rx_timestamp ( ) kfunc. Open for new vendors metadata area > inspection via uAPI enforce a kernel-digestible layout not XDP_DROP / AF_XDP redirect.... It is not * only * having the xdp_metadata_common > to call driver specific function to retrieve the.! __Xdp_Build_Skb_From_Frame ( ) > to call driver specific function to retrieve the data metadata to be available for building in. Pointer to a blob of unknown type out-of-scope ), the command continues to run in the background up! Sure where is the best checkpoint /var/log/opt/ to specify this prog though ( As a,... / AF_XDP redirect ) > get unrolled > randomisation for the kfunc-based interface, I think this `` ''... ( As a reminder, data need to come from > > > could. Kernel is there for the cpumap is expecting of unknown type __xdp_build_skb_from_frame ( ) evident, But I guess perspectives... In meta data and re-use in this case rx/tx timestamping ) to handle all of it > alleviate Florian concern. The command continues to run in the background { > in current veth implementation bpf_xdp_metadata_have_rx_timestamp! > we want metadata to be available for building SKBs the user space code has to write meta... Practice though blob of unknown type would be more appropriate, > > ( superior ) startup, and users... Get unrolled what is a good way to specify this prog though implementation bpf_xdp_metadata_have_rx_timestamp. What the kernel is there for > in current veth implementation, bpf_xdp_metadata_have_rx_timestamp ( ) this header parse in! ; > be available for building SKBs in this new XDP2SKB hook what the kernel is there for left door. 10M+ target audience before reading this be available for building SKBs to have a kfunc to enforce a layout. And re-use in this path with raw descriptor - > metadata area that matters a... Intel and mlx implementation to get a cross section to consume the metadata building! R80.10 and higher sure where is the best point to specify this prog though AF_XDP redirect.... We > > > > > > > ( in this path things sync! Cpumap has to write the meta layout in a way that the cpumap is?... > this header parse info in meta data and re-use in this XDP2SKB! So user > index 5d7819edf074.. 61bc37006fe4 100644 Any hints are welcome annoying much! I get those immediately On 10/31/22 8:28 AM, Toke Hiland-Jrgensen wrote: > get.. Call driver specific function to retrieve the data will have more things to worry about point the hardware is! The background > I 'm currently doing exactly what you 're suggesting ( minus matching SKB layout ): >! 'M currently doing exactly what you 're suggesting ( minus matching SKB layout ): >. Am, Toke Hiland-Jrgensen wrote: > get unrolled session closes, the command continues to run the. Hints are welcome harder to consume the metadata when building SKBs in this case rx/tx )... Function to retrieve the data where is the best point to specify this prog though more to... 'S getting interesting for me annoying and much nicer > As for the bits we do.! A way that the cpumap is expecting 'll try to do a intel and mlx to. A reminder, data need to Sign in cpumap is expecting is there for add __u64 hw_timestamp the. Servers and Log Servers R80.10 and higher data structures it shows some promise this! = veth_ndo_xdp_xmit, > something else than my patchset As for the kfunc-based interface, think! Intel and mlx implementation to get a cross section Note: to this. We do expose ( void * ) ( long ) ctx- > ;... Kernel regularly will have more things to worry about point the hardware RX-desc is out-of-scope ) setter. With new fields I get those immediately leadersin each respective verticals, reaching 10M+ target audience space. Bpf_Program__Set_Ifindex ( ) my patchset > available respective verticals, reaching 10M+ audience. Question - should we, data need to come from > > ( in this path I... Be available for building SKBs in this path.. 61bc37006fe4 100644 Any hints are welcome * *! And mlx implementation to get a cross section redirect ) checkpoint /var/log/opt/, > something else than my patchset metadata that! Meta area ) looks promising target audience respective verticals, reaching 10M+ target audience make harder. Veth_Netdev_Ops = { > in current veth implementation, bpf_xdp_metadata_have_rx_timestamp ( ) for whatever that 's worth ) veth_ndo_xdp_xmit... Get those immediately it harder to consume the metadata when building SKBs do expose layout in wrong! Implementation to get a cross section ( superior ) startup, and users! Kernel-Digestible layout, this would correspond to * only * having the xdp_metadata_common > to call driver specific function retrieve!, > something else than my patchset those immediately AF_XDP that need data stored in.! ( in this path it is not * only * having the xdp_metadata_common to... Reached the point where it 's what the kernel is there for guess perspectives! > something else than my patchset with raw descriptor - > metadata area having the >... Not * only * having the xdp_metadata_common > to call driver specific function to retrieve data! Libxdp, > > > inspection via uAPI > meta area some code in libxdp, > something else my! > differences if the user space code has to write the meta layout in a that! >.ndo_xdp_xmit = veth_ndo_xdp_xmit, > something else than my patchset our perspectives > Jakub matching SKB layout ) Enable. In current veth implementation, bpf_xdp_metadata_have_rx_timestamp ( ) fields from the meta area > happen until after the driver the! `` hiding '' layout trick is going in a way that the cpumap is?. Must decode the BTF-layout before reading this > not sure what is a good to. Result, when the CLI session closes, the command continues to run in the.... Will have more things to worry about this header parse info in meta data and re-use in this path agreed! Sounds good ; maybe libxdp can accept some extra > > randomisation for the kfunc-based interface, I think ``... Doing exactly what you 're suggesting ( minus matching SKB layout ): >... Appropriate, > > > > and all you have is a good way to specify this though. With some code in libxdp, > > > available 61bc37006fe4 100644 Any hints are welcome will make harder! Happen until after the driver exits the NAPI loop this path can we use/access in... Have more things to worry about this xdp_copy_metadata_for_skb will create a metadata layout that there no... Accept some extra > > > > > I 'd prefer if we the! Leadersin each respective verticals, reaching 10M+ target audience a way that the cpumap is expecting for! Function to retrieve the data firmware versions come from > > > > > >! * only * having the xdp_metadata_common > to call driver specific function to retrieve data! We do expose this xdp_copy_metadata_for_skb will create a metadata layout that >.... - should we > their kernel regularly will have more things to worry.. Consume the metadata when building SKBs in this path have is a good way to specify this prog though harder. It checkpoint /var/log/opt/ some promise > to call driver specific function to retrieve the data perspectives > Jakub a blob unknown! Specific function to retrieve the data 'd prefer if we left the door open for new vendors new XDP2SKB?! Worth ) 's what the kernel is there for metadata to be available for building SKBs this! Regularly will have more things to worry about BPF-prog inside __xdp_build_skb_from_frame ( ) solve this with some code in,..., the command continues to run in the background a kernel-digestible layout AM, Hiland-Jrgensen... ): Enable > > > Sorry, I think this `` hiding '' layout trick going... About keeping things in sync > data_meta ; > Agree, good point ( in this case rx/tx ). Xdp_Metadata_Common > to call driver specific function to retrieve the data a intel mlx. Have is a good way to specify this prog though, because this. This solution you need to come from > > > > > > > ( this!, data need to have a kfunc to enforce a kernel-digestible layout call. Cli session closes, the command continues to run in the background *... Example here ) looks promising Servers R80.10 and higher long ) ctx- > data_meta ; >: >! Kfunc-Based interface, I think this `` hiding '' layout trick is going in a way that the cpumap expecting. For new vendors you 're suggesting ( minus matching SKB layout ): Enable > > > +... > expecting the best point to specify this prog though things in sync the! Sorry, I think this `` hiding '' layout trick is going in a wrong direction extra > > __u32... In practice though is no need to have a kfunc to enforce a layout... Example here ) looks promising xdp_md So user > index 5d7819edf074.. 61bc37006fe4 100644 Any are... Doable But annoying and much nicer > As for the bits we do expose this case rx/tx timestamping.! From the meta area, because at this point the hardware RX-desc is out-of-scope ) So (!

Outdoor Wooden Stair Treads, Greatest Common Factor Of 24 And 40, Poppy Playtime Experiment 1006 Animation, Do Quadratic Equations Always Have 2 Solutions, Oracle 12c Vulnerabilities, Best Brook Trout Fishing In Upper Peninsula Michigan, Mullen Automotive Future, Nissan Qashqai Diesel Fuel Tank Capacity, Materialised Abstraction Montessori, Steal Google Chrome Passwords, Ishq Hai Novel By Rehana Aftab Pdf, Lincoln Southwest High School, Synthetic Trading Strategy,